Renewable  and  Sustainable  Energy  Reviews  15  (2011)  3918-3933 


ELSEVIER 


Contents  lists  available  at  ScienceDirect 

Renewable  and  Sustainable  Energy  Reviews 

journal  homepage:  www.elsevier.com/locate/rser 


Towards  integrated  sustainability  assessment  for  energetic  use  of  biomass:  A 
state  of  the  art  evaluation  of  assessment  tools 

V.  Buytaert* 1 2 3’*,  B.  Muysb,  N.  Devriendt3,  L.  Pelkmans3,J.G.  Kretzschmarc,  R.  Samsonc 

a  VITO,  Flemish  Institute  for  Technological  Research,  Boeretang  200,  B-2400  Mol,  Belgium 

b  Division  Forest,  Nature  and  Landscape,  Katholieke  Universiteit  Leuven,  Celestijnenlaan  200E,  Box  241 1,  B-3001  Leuven,  Belgium 
c  Department  of  Bioscience  Engineering,  Universiteit  Antwerpen,  Groenenborgerlaan  171  V,  B-2020  Antwerpen,  Belgium 


ARTICLE  INFO 


ABSTRACT 


Article  history: 

Received  5  January  201 1 
Accepted  5  July  201 1 
Available  online  6  August  201 1 


Keywords: 

Bioenergy 

Criteria  and  Indicators 

Life  Cycle  Assessment 

Cost  Benefit  Analysis 

Environmental  Impact  Assessment 

Exergy  Analysis 

System  Perturbation  Analysis 


Biomass  is  expected  to  play  an  increasingly  significant  role  in  the  ‘greening’  of  energy  supply.  Neverthe¬ 
less,  concerns  are  rising  about  the  sustainability  of  large-scale  energy  crop  production.  Impacts  must  be 
assessed  carefully  before  deciding  whether  and  how  this  industry  should  be  developed,  and  what  tech¬ 
nologies,  policies  and  investment  strategies  should  be  pursued.  There  is  need  for  a  comprehensive  and 
reliable  sustainability  assessment  tool  to  evaluate  the  environmental,  social  and  economic  performance 
of  biomass  energy  production.  This  paper  paves  the  way  for  such  a  tool  by  analysing  and  comparing 
the  performance  and  applicability  of  a  selection  of  existing  tools  that  are  potentially  useful  for  sustain¬ 
ability  assessment  of  bioenergy  systems.  The  selected  tools  are:  Criteria  And  Indicators  (C&I),  Life  Cycle 
Assessment  (LCA),  Environmental  Impact  Assessment  (EIA),  Cost  Benefit  Analysis  (CBA),  Exergy  Analysis 
(EA)  and  System  Perturbation  Analysis  (SPA).  To  evaluate  the  tools,  a  framework  was  constructed  that 
consists  of  four  evaluation  levels:  sustainability  issues,  tool  attributes,  model  structure,  area  of  applica¬ 
tion.  The  tools  were  then  evaluated  using  literature  data  and  with  the  help  of  a  Delphi  panel  of  experts. 
Finally,  a  statistical  analysis  was  performed  on  the  resulting  data  matrix  to  detect  significant  differences 
between  tools.  It  becomes  clear  that  none  of  the  selected  tools  is  able  to  perform  a  comprehensive  sus¬ 
tainability  assessment  of  bioenergy  systems.  Every  tool  has  its  particular  advantages  and  disadvantages, 
which  means  that  trade-offs  are  inevitable  and  a  balance  must  be  found  between  scientific  accuracy  and 
pragmatic  decision  making.  A  good  definition  of  the  assessment  objective  is  therefore  crucial.  It  seems 
an  interesting  option  to  create  a  toolbox  that  combines  procedural  parts  of  C&I  and  EIA,  supplemented 
with  calculation  algorithms  of  LCA  and  CBA  for  respectively  environmental  and  economic  sustainability 
indicators.  Nevertheless,  this  would  require  a  more  comprehensive  interdisciplinary  approach  to  align 
the  different  tool  characteristics  and  focuses. 
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1.  Introduction 

2.2.  Current  state  of  sustainability  assessment 

The  concept  of  sustainability  has  evolved  considerably  since  it 
was  mentioned  for  the  first  time  in  the  forestry  sector  of  the  18th 
century  [1-8].  Nowadays,  the  standard  definition  of  sustainable 
development  provided  by  the  Brundtland  Commission  “to  ensure 
that  it  meets  the  needs  of  the  present  without  compromising  the 
ability  of  future  generations  to  meet  their  own  needs”  [9]  is  gener¬ 
ally  recognised  as  a  standard  and  a  starting  point  for  most  who  set 
out  to  define  the  concept.  The  concept  is  often  depicted  schemat¬ 
ically  using  three  overlapping  circles  for  the  target  dimensions  of 
environment,  economy  and  society,  to  which  might  be  added  an 
institutional  dimension  [10].  More  detailed  discussions  of  differ¬ 
ent  concepts  of  sustainability  can  be  found  in  various  publications 
[11-13]. 

For  the  transition  towards  a  more  sustainable  society,  com¬ 
prehensive  but  efficient  and  reliable  assessment  tools  are  needed. 
Devuyst  et  al.  [14]  define  sustainability  assessment  as  “a  tool  that 
can  help  decision-makers  and  policy-makers  decide  which  actions 
they  should  or  should  not  take  in  an  attempt  to  make  society  more 
sustainable.”  Following  Ness  et  al.  [15]  the  purpose  of  sustainabil¬ 
ity  assessment  is  to  provide  decision-makers  with  an  evaluation 
of  global  to  local  integrated  nature-society  systems  in  short  and 
long  term  perspectives  in  order  to  assist  them  to  determine  which 
actions  should  or  should  not  be  taken  in  an  attempt  to  make  society 
more  sustainable.  Based  on  the  four  dimensions  of  sustainabil¬ 
ity,  several  comprehensive  frameworks  for  integrated  assessment 
have  been  developed  [16-18].  In  1996,  a  group  of  researchers  have 
established  a  set  of  practical  guidelines  in  the  form  of  overarch¬ 
ing  principles  that  provide  a  link  between  theory  and  practice  for 
the  whole  assessment  process  from  system  design  and  identifi¬ 
cation  of  indicators,  through  field  measurements  and  compilation 
to  interpretation  and  communication  of  results,  which  are  known 
as  the  “Bellagio  Principles”  [19].  In  the  last  decade,  a  number  of 
concepts  have  been  developed  that  aim  at  upgrading  the  opera¬ 
tionalisation  of  the  vision  of  sustainability.  This  resulted  in  a  diverse 
field  of  approaches  that  claim  that  they  can  be  used  for  assessing 
sustainability  in  many  different  sectors  [15,18-27].  Sustainability 
assessment  has  increasingly  become  associated  with  the  family  of 
impact  assessment  tools  such  as  Life  Cycle  Assessment,  Ecological 
Footprint,  Environmental  Impact  Assessment  and  Strategic  Envi¬ 
ronmental  Impact  Assessment  [8,28]. 

2.2.  Problem  statement ,  aim  and  approach 

In  this  research,  the  assessment  focus  is  on  the  bioenergy  sector. 
Worldwide,  the  potential  of  biomass  to  contribute  to  reductions 
in  greenhouse  gas  (GHG)  emissions,  to  improved  energy  security 
and  to  rural  diversification  and  development  is  recognised  [16]. 
Several  bioenergy  policies  such  as  the  Brazilian  Biofuel  Law,  the 
Indian  Biofuel  Policy,  the  U.S.  Renewable  Fuel  Standard,  and  Euro¬ 


pean  policies  such  as  the  Renewable  Energy  Directive,  Strategy 
on  Biofuels  and  the  Biomass  Action  Plans  push  the  implementa¬ 
tion  of  energy  derived  from  biomass  in  order  to  reduce  fossil  fuel 
dependency  or  to  meet  greenhouse  gas  reduction  targets.  These 
policies  are  a  source  of  increasing  concern  due  to  questions  over 
the  sustainability  of  large-scale  bioenergy  crop  production.  Poten¬ 
tial  negative  impacts  include  direct  and  indirect  land  use  change 
[29-32  ]  and  biodiversity  loss  [33,34],  availability  of  water  resources 
[35-38],  rising  agricultural  commodity  prices  and  threats  to  food 
security  [39-44].  These  risks  have  to  be  weighed  against  potential 
benefits  such  as  improved  greenhouse  gas  balance,  employment 
and  income  generation,  rural  development,  conversion  of  conven¬ 
tional  industries  and  increased  security  of  energy  supply  [45-47]. 
Therefore,  the  environmental,  social  and  economic  impacts  of 
bioenergy  development  needs  to  be  assessed  carefully  before 
deciding  whether  and  how  this  industry  should  be  developed,  and 
what  technologies,  policies  and  investment  strategies  should  be 
pursued.  In  this  context,  some  governments  and  institutions  started 
developing  sustainability  tools  and  standards  to  evaluate  the  envi¬ 
ronmental,  social  and  economic  performance  of  biomass  energy 
production.  At  the  same  time,  companies  are  willing  to  integrate 
more  sustainable  strategies  in  their  energy  management  and  are 
looking  for  practical  and  reliable  sustainability  assessment  tools 
to  assess  different  energy  system  possibilities.  However,  the  high 
variability  in  biomass  sources,  conversion  technologies  and  con¬ 
texts  complicate  such  assessments.  A  variety  of  assessment  tools 
for  sustainability  evaluation  of  production  systems  has  been  devel¬ 
oped  in  the  past,  each  showing  a  different  procedure  and  field  of 
application  [48]. 

The  general  aim  of  this  paper  is  to  contribute  to  the  development 
of  a  comprehensive,  yet  practical  and  reliable  tool  for  sustainability 
assessment  of  bioenergy  systems.  Since  many  tools  currently  exist, 
there  is  more  of  a  need  to  highlight  complementarities  or  possibili¬ 
ties  for  integration  rather  than  to  generate  new  tools.  Therefore,  the 
specific  goal  of  this  paper  was  to  make  a  comparative  evaluation  of 
a  selection  of  existing  tools  to  which  degree  these  tools  are  able 
to  incorporate  the  different  dimensions  of  sustainability  of  bioen¬ 
ergy  systems.  The  research  approach  is  presented  in  Fig.  1.  First,  an 
evaluation  framework  is  constructed  in  which  the  tools  are  evalu¬ 
ated.  In  a  next  step,  the  evaluation  of  the  selected  tools  is  made.  The 
resulting  data  matrix  is  then  statistically  analysed  to  detect  signif¬ 
icant  differences  between  tools  and  as  a  conclusion,  strengths  and 
weaknesses  of  the  tools  are  highlighted  and  possibilities  for  their 
integration  in  a  more  comprehensive  tool  are  made. 

2.  Materials  and  methods 

2.2.  Selection  of  existing  assessment  methods 

A  wide  range  of  sustainability  assessment  methods  has  been 
developed  in  recent  years  hence  some  initial  selection  was  needed. 
Our  selection  strategy  was  twofold.  On  the  one  hand  we  chose  some 
well-known  and  commonly  used  but  conceptually  very  different 
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Fig.  1.  Flow  chart  of  the  tool  evaluation  process. 


tools  for  sustainability  assessment:  Criteria  And  Indicators  (C&I), 
Life  Cycle  Analysis  (LCA),  Environmental  Impact  Assessment  (EIA) 
and  Cost  Benefit  Analysis  (CBA).  On  the  other  hand  we  added  two 
rather  uncommon  but  conceptually  promising  tools.  Exergy  Anal¬ 
ysis  (EA)  is  a  method  in  thermodynamics  able  to  account  material 
and  energy  flows  all  in  the  same  unit.  System  Perturbation  Analysis 
(SPA)  is  a  tool  that  performs  an  accountancy  evaluation  after  per¬ 
turbation  of  a  system  (e.g.  increase  in  bioenergy  production)  and 
compensation  of  the  disturbances.  SPA  was  specifically  designed  for 
the  Belgian  system.  The  result  of  the  selection  is  a  mixture  of  tools 
with  varying  scopes  and  assessment  strategies,  focussing  on  dif¬ 
ferent  systems  (products,  services,  projects  and/or  economies)  and 
various  impacts.  A  short  introduction  to  each  tool  is  given  in  Box  1 


Box  1 

Criteria  and  indicators  (C&I)  can  be  used  for  a  wide  range  of 
applications  such  as  eco-certification  at  the  management  unit 
level  [49],  policy  evaluation  at  the  regional  or  national  level 
[50,51],  or  as  a  generic  assessment  tool  for  specific  environ¬ 
mental,  social,  economic  or  institutional  sustainability  issues. 
The  use  of  C&I  is  a  common  way  to  describe  and  monitor  com¬ 
plex  systems,  and  to  provide  information  to  decision  makers 
and  the  public.  Some  well-known  sustainability  criteria  initia¬ 
tives  in  the  field  of  bioenergy  [52-54]  are  the  criteria  of  the 
Cramer  Commission  [17,55],  the  Renewable  Transport  Fuels 
Obligation  [56-58],  the  European  Renewable  Energy  Directive 
[59],  the  Round  Table  for  Sustainable  Palm  Oil  [60,61],  the 
Round  Table  on  Responsible  Soy  [62],  the  Basel  Criteria  for 
Responsible  Soy  Production  [63],  the  Roundtable  on  Sustain¬ 
able  Biofuels  [64-66],  the  Forest  Stewardship  Council  FSC  [67] 
and  the  Programme  for  the  Endorsement  of  Forest  Certification 
PEFC  [68,69]. 

A  well-established  method  is  Life  Cycle  Assessment  (LCA). 
LCA  has  been  used  in  varying  forms  over  the  past  35 years 
to  evaluate  the  environmental  impacts  of  a  product  or  a  ser¬ 
vice  throughout  its  life  cycle.  It  is  an  approach  that  analyses 
real  and  potential  pressure  that  a  product  has  on  the  envi¬ 
ronment  during  its  whole  life  cycle  from  cradle  to  grave, 
including  raw  material  acquisition,  production  process,  use, 
and  disposal  of  the  product  [70].  The  International  Standards 
Organisation  (ISO)  has  established  guidelines  and  principles 
for  LCA  that  have  been  further  interpreted  and  developed 
by  many  authors  [25,71-76].  LCA  results  provide  information 
for  decisions  regarding  product  development  and  ecodesign, 
production  system  improvements,  and  product  choice  at  the 
consumer  level.  Numerous  Life-cycle  Assessments  have  been 
performed  in  the  bioenergy  field  [77-88],  as  well  as  for  a  mul¬ 
titude  of  other  product  and  service  areas. 

Environmental  Impact  Assessment  (EIA)  has  been  used  since 
1960s  for  evaluating  potential  environmental  impacts  -  con¬ 
sidering  natural,  social  and  economic  issues  -  that  a  proposed 
project  may  have,  with  the  aim  to  reduce  its  negative 


effects  [89].  The  purpose  of  the  assessment  is  to  decide 
whether  to  proceed  with  the  project  and  to  identify  impact 
mitigating  measures.  The  International  Association  for  Impact 
Assessment  (IAIA)  defines  an  Environmental  Impact  Assess¬ 
ment  as  the  process  of  identifying,  predicting,  evaluating  and 
mitigating  the  biophysical,  social,  and  other  relevant  effects  of 
development  proposals  prior  to  major  decisions  being  taken 
and  commitments  made  [90].  In  the  EU,  a  directive  that  made 
EIA  compulsory  for  proposed  public  and  private  projects  (e.g. 
construction  projects)  that  are  likely  to  have  environmental 
impacts  was  introduced  in  1985  [91,92].  EIA  has  also  been  intro¬ 
duced  in  the  legislation  in  many  other  countries  [93].  Due  to 
such  legal  requirements,  there  are  strict  guidelines  for  the  EIA 
process  in  the  EU  and  other  countries  [89].  In  literature,  no  EIA 
studies  were  found  for  bioenergy  projects  in  particular. 

Cost  Benefit  Analysis  (CBA)  is  an  applied  welfare  economics 
tool  with  roots  reaching  back  to  the  early  20th  century.  It  is 
used  for  evaluating  public  or  private  investment  proposals  by 
weighing  the  costs  of  the  project  against  the  expected  ben¬ 
efits.  In  the  realm  of  sustainability  assessment,  CBA  can  be 
an  effective  tool  for  weighing  the  social  costs  and  benefits  of 
different  alternatives  in  connection  with  e.g.  energy  and  trans¬ 
ports  [94].  It  is  this  aspect  of  measuring  expected  benefits,  or 
placing  monetary  units  on  the  benefits,  e.g.  biodiversity,  that 
is  often  a  challenge  within  CBA  [95].  A  number  of  CBA  studies 
were  found  regarding  bioenergy  projects  [96-100]. 

Exergy  Analysis  (EA)  is  a  powerful  thermodynamic  account¬ 
ing  technique  for  assessing  and  improving  the  efficiency  of 
processes,  devices  and  systems,  as  well  as  for  enhancing  envi¬ 
ronmental  and  economic  performance.  As  a  multidisciplinary 
concept,  exergy  applications  are  observed  in  various  fields, 
including  mechanical  and  chemical  engineering  as  well  as  eco¬ 
nomics,  management,  physics  and  biology.  Consequently,  EA 
is  used  increasingly  by  industries  and  governments  through¬ 
out  the  world,  particularly  with  the  aim  of  improving  energy 
sustainability.  In  depth  explanations  of  the  concept  and  appli¬ 
cation  of  exergy  analyses  are  delivered  by  many  scientist 
[101-109],  and  examples  of  exergy  assessment  related  to 
the  bioenergy  field  can  be  found  in  [110-112,102,113,114]. 
A  new  method  called  System  Perturbation  Analysis  (SPA)  was 
recently  developed  by  the  Free  University  of  Brussels  (VUB) 
[115].  SPA  differs  from  a  conventional  Life  Cycle  Analysis  (LCA) 
mainly  because  it  looks  to  alternations  in  resource  balances 
(so-called  perturbations)  within  a  specific  geographical  system 
and  the  resulting  effects,  rather  than  comparing  life-cycle  tra¬ 
jectories.  Therefore  SPA  is  able  to  identify  the  best  usage  of 
limited  resources  such  as  hectares,  wood  waste  or  imports, 
in  terms  of  fossil  energy  savings  or  GHG  emissions  within  a 
given  system.  Comparative  results  of  such  a  System  Pertur¬ 
bation  Analysis  for  the  use  of  wood  for  transport,  heat  and 
power  applications  are  presented  in  the  study  of  Bram  et  al. 
[115]  and  results  for  biomass  in  a  natural  gas  combined  cycle 
power  plant  can  be  found  in  the  study  of  Delattin  et  al.  [116]. 
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2.2.  Step  1:  construction  of  an  evaluation  framework  for 
sustainability  assessment  tools 

The  selected  tools  represent  different  ways  of  looking  at  a  given 
assessment  challenge  and  they  convey  different  information  to  the 
decision  process.  Each  tool  collects,  structures  and  evaluates  infor¬ 
mation  about  specific  issues  and  presents  it  in  a  specific  way.  Which 
specific  tool  will  be  used  in  a  specific  situation  depends  on  the  gen¬ 
eral  conceptual  view  of  the  decision  maker,  the  context  for  which  to 
take  a  decision  and  the  specific  objectives  of  the  assessment  [117]. 
In  order  to  make  a  clear  evaluation  of  the  selected  tools  and  to  com¬ 
pare  their  characteristics  in  terms  of  objectives,  focus  and  approach, 
an  evaluation  framework  is  constructed.  The  definition  of  an  eval¬ 
uation  framework  depends  on  the  specific  aim  of  a  study,  which 
in  our  case  is  the  development  of  a  comprehensive  yet  practical 
and  reliable  method  for  sustainability  assessment  of  bioenergy  sys¬ 
tems.  ‘Comprehensive’  means  that  as  many  environmental,  social, 
economic  and  institutional  sustainability  issues  as  possible  should 
be  considered  in  the  tool.  A  ‘practical’  tool  makes  demands  on  the 
effective  use  of  the  tool  and  is  related  to  the  amount  of  data  that 
are  needed,  which  procedure  is  applied,  how  the  results  are  repro¬ 
duced,  and  in  which  context  the  tool  can  be  used.  ‘Reliable’  requires 
a  transparent  and  science  based  model  structure  that  takes  all 
inputs,  outputs,  interactions  and  interdependencies  at  each  stage 
of  the  bioenergy  system  into  account  [  1 1 8  ].  All  these  requirements 
which  determine  whether  a  tool  is  suitable  or  not  are  addressed  in 
the  evaluation  framework,  using  four  levels  of  comparison:  (i)  sus¬ 
tainability  issues;  (ii)  tool  attributes;  (iii)  model  structure;  and  (iv) 
area  of  application.  The  first  two  levels  could  be  analysed  numer¬ 
ically  by  a  Delphi  panel  of  experts,  whereas  the  last  two  levels, 
which  are  descriptive,  are  evaluated  based  on  literature  review. 
These  four  levels  of  comparison  of  the  evaluation  framework  are 
further  elaborated  in  the  following  paragraphs. 

2.2.1.  Issues  of  sustainability 

An  important  part  in  determining  the  ability  of  a  tool  to 
serve  as  a  sustainability  assessment  tool  for  bioenergy  sys¬ 
tems  is  to  check  which  sustainability  issues  are  addressed 
by  the  tool.  For  this  aim,  a  comprehensive  list  of  relevant 
sustainability  issues  was  constructed,  based  on  sustainability 
requirements  for  bioenergy  systems  mentioned  in  literature, 
e.g.  [2,17,52,56,57,59,62,63,119-122].  The  resulting  list,  organised 
along  the  four  dimensions  of  sustainability,  is  shown  in  Table  1.  It 
was  chosen  not  to  go  in  too  much  detail  to  keep  the  list  operational. 
A  few  indicators  are  added  as  an  example. 

2.2.2.  Tool  attributes 

Assessment  tools  ideally  have  a  number  of  characteristics  which 
facilitate  their  application  with  respect  to  the  use  of  data,  the  fol¬ 
lowed  methodological  procedure,  the  reproduction  of  the  results, 
and  finally,  the  use  of  the  tool  in  general.  In  this  second  level  of  the 
evaluation  framework,  it  is  tried  to  consider  a  large  range  of  facili¬ 
tating  attributes  that  a  sustainability  assessment  tool  for  bioenergy 
systems  might  have.  A  list  of  attributes  was  made  using  informa¬ 
tion  about  tool  attributes,  found  in  various  literature  resources 
[1,48,122],  complemented  with  own  expertise.  The  result  can  be 
found  in  Table  2,  in  which  attributes  are  given  with  respect  to  data 
used  in  the  tool,  tool  procedure,  tool  results,  and  use  of  the  tool. 

2.2.3.  Overall  model  structure 

Literature  review  of  assessment  frameworks,  tools  and  compar¬ 
ison  studies  has  demonstrated  that  tools  can  be  categorised  based 
on  several  features  with  regard  to  the  overall  model  structure  e.g. 
[11,1 5,77,95,1 23-1 26].  Of  all  the  features  encountered  in  literature, 
those  that  seemed  relevant  for  sustainability  assessment  of  bioen¬ 
ergy  systems  were  withheld  for  the  third  level  of  the  evaluation 


Table  1 

Set  of  sustainability  issues  for  bioenergy  systems  (examples  of  indicators  between 
brackets). 

Sustainability  issues  and  some  of  their  indicators 
Environmental  dimension 
Air  quality  (emissions  of  GHG,  nitrogen,  POP’s) 

Soil  quality  (nutrient  balances,  accumulation  of  heavy  metals,  soil 
compaction  and  erosion) 

Land  use  (amount  of  land  used,  leakage  effects) 

Water  quality  (water  pollution,  eutrophication,  pesticide  residues) 

Use  of  ground  and  surface  water 

Biodiversity  and  preservation  of  sensitive  ecosystems  (use  of  GMO’s/exotic 
species,  preservation  of  habitats  and  high  conservation  value  forests, 
avoidance  of  pollution  of  neighbouring  ecosystems) 

Deforestation 

Desertification  and  drought 

Landscape  view  (landscape  variation,  conservation  of  typical  elements) 

Use  of  agrochemicals 

Waste  management  (minimization,  sorting,  disposal,  recycling) 

Use  of  energy  (energy  balance) 

Use  of  fossil  resources 
Social  dimension 

Labour  conditions  (freedom  of  association,  collective  bargaining,  minimum 
wages,  discrimination) 

Protection  of  human  safety  and  health  (protection  of  human  health,  safe  and 
healthy  work  environment) 

Protection  of  human  rights  (rights  of  children,  women,  indigenous  people, 
discrimination) 

Access  to  resources  ensuring  adequate  quality  of  life  (access  to  potable 
water,  sanitary  facilities,  adequate  housing,  education  and  training, 
transportation,  and  health  services) 

Food  and  energy  supply  safety  (quantity  and  quality  of  food  available, 
energy  supply) 

Capacity  building  (building  and  use  of  local  labour  and  skills) 

Democratic  participation  (stakeholder  involvement) 

Protection  of  property  rights  and  user  ownership  (land  tenure  conflicts, 
equitable  land  ownership) 

Fair  trade  conditions  (transparency  and  accountability  of  negotiations,  fair 
and  equal  remuneration) 

Social  acceptance  (acceptance  of  the  business  by  producer,  consumer  and 
local  population) 

Economic  dimension 

Viability  of  the  business  (net  present  value,  minimisation  of  costs,  adequate 
funding) 

Long  term  perspective  (long-term  commitments,  contracts  and 
management  plans) 

Reliability  of  resources  (security  of  supply) 

Reliability  of  the  used  technology  (adequately  proven  technology) 

Risk  minimisation 

Strength  and  diversification  of  local  economy 
Reliability  of  energy 

No  blocking  of  other  desirable  developments 
Institutional  dimension 

In  respect  with  local,  national  and  international  policy  and  legislation 
Institutional  capacity 
Institutional  well-being 
Environmental  management 


framework  of  this  study.  After  a  short  explanation  of  a  few  general 
tool  aspects  such  as  tool  description,  tool  concept,  object  of  analy¬ 
sis,  type  of  comparison,  degree  of  standardisation  or  harmonisation, 
and  frequency  of  use,  some  more  specific  methodological  aspects 
are  addressed:  time  frame,  spatial  scale,  concept  focus,  quantita¬ 
tive  or  qualitative  character,  reproduction  of  results,  assignment 
of  weight  scores,  and  sensitivity  and  uncertainty  analysis.  A  short 
description  of  the  different  aspects  follows. 

The  tool  description  describes  the  purpose  of  the  tool,  the  reason¬ 
ing  behind  the  development  of  the  tool.  The  tool  concept  gives  an 
overview  of  the  methodology  followed  or  the  different  steps  during 
application  of  the  tool.  The  object  of  analysis  specifies  which  object 
is  being  analysed  by  the  tool,  e.g.  a  product,  a  function  or  an  econ¬ 
omy.  Most  tools  include  some  type  of  comparison,  either  between 
different  alternatives,  or  within  a  studied  system,  or  against  a  ref¬ 
erence.  It  differs  from  tool  to  tool  and  is  often  related  to  the  object 
analysed  in  the  decision  process  [123].  Also  the  degree  ofstandardis- 
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Table  2 

List  of  attributes  used  to  describe  the  characteristics  of  the  evaluated  tools. 


Tool  attribute 

Attribute  explanation 

Data 

Availability 

Data  needed  to  apply  the  tool  are  readily 
available 

Need 

Balanced  amount  of  data  needed  to  apply  the 
tool 

Type 

Preference  for  quantitative  (metric)  data  input 

Integration 

Ability  to  combine  descriptive  and  quantitative 
data 

Correctness 

Correctness  of  data  used 

Procedure 

Flexibility 

Model  procedure  is  dynamic,  can  easily  be 
adapted  or  complemented 

Coverage 

Degree  to  which  multiple  sustainability  issues 
of  the  four  dimensions  are  integrated 

Objectivity 

Model  is  not  susceptible  to  the  subjectivity  of 
the  executor 

Transparency 

Transparency  of  the  evaluation  procedure 

Validation 

Ability  to  validate  evaluation  results 

Time  scale 

Ability  to  process  dynamic  data 

State-of-the-art 

Degree  to  which  the  tool  is  operational 

Results 

Univocity 

Results  are  consistent,  no  contradictions  in  the 
results 

Simplicity 

Results  are  possible  to  understand  by 
stakeholders,  no  expert  knowledge  required 

Type 

Results  are  preferably  quantitative 

Clarity 

Results  are  compact  and  clear,  no  long  lists  or 
extensive  reports 

Communication 

Results  are  represented  graphically,  easy  to 
interpret 

Use 

Flexibility 

Ability  to  integrate  inputs  of  the  stakeholders 

Accessibility 

Possibility  to  interpret  results  without  expert 
intervention 

Application  cost 

Low  cost  to  execute  the  evaluation  task 

Application  field 

Ability  to  execute  evaluation  tasks  on  different 
levels  (micro,  macro, . . .) 

Time  consumption 

Low  time  demand  to  execute  the  evaluation 
task 

Handiness 

The  model  is  easy  to  use 

Assistance 

Availability  of  experts  to  render  assistance 

ation  or  harmonisation  is  an  important  aspect,  even  if  many  aspects 
are  under  continuous  development  [95].  If  different  users  apply 
the  tool  in  separate  ways,  evolution  will  make  it  harder  to  keep  the 
tool  homogenous  in  the  future  and  to  compare  or  share  data  and 
results.  Another  aspect  is  the  frequency  of  use,  which  gives  informa¬ 
tion  about  how  often  the  tool  is  used,  and  sometimes  also  where  it 
is  used.  The  timeframe  both  indicates  the  time  perspective  applied 
to  the  object  analysed  which  can  be  retrospective  or  prospective,  as 
well  as  the  time  horizon  of  the  object  itself  and  the  related  impacts. 
Approaches  mainly  used  in  the  context  of  accounting,  tracking  and 
monitoring  on-going  (e.g.  year  on  year)  progress  can  be  called  retro¬ 
spective,  while  those  used  to  predict  future  situations  can  be  called 
prospective  [123].  The  time  horizon  indicates  whether  the  tool  is  a 
snapshot  in  time,  or  whether  it  considers  the  whole  life  time  of  a 
product,  project  or  process  and/or  their  impacts.  The  spatial  scale  is 
related  to  the  spatial  boundaries  and  horizon  of  the  object  analysed. 
Spatial  boundaries  can  be  the  boundaries  of  a  country  or  town,  but 
also  boundaries  surrounding  a  section  in  a  product  chain,  or  the 
boundary  between  nature  and  the  human  system  [95].  The  spatial 
horizon  may  include  a  single  site,  many  sites,  or  no  sites  e.g.  in  case 
of  product  design.  The  spatial  scale  can  therefore  be  categorised  as 
site  specific  focus  and  non-  or  multi-site  focus  [123].  The  concept 
focus  indicates  whether  a  tool  emphasises  procedure  or  focuses  on 
the  construction  of  a  defined  type  of  computational  algorithm.  Pro¬ 
cedure  aims  at  describing  the  best  way  to  reach  a  decision,  whereas 
algorithms  are  aimed  at  finding  the  better  decision  [123].  Tools  can 
have  a  purely  quantitative  character  or  rather  qualitative,  or  both 


quantitative  and  qualitative.  Under  reproduction  of  results,  infor¬ 
mation  is  given  about  how  the  results  are  presented,  whether  they 
are  listed  with  different  units,  or  aggregated  in  one  single  outcome. 
Another  aspect  that  is  regarded,  is  the  possibility  within  the  tool  to 
assign  weight  scores  and  how  this  is  performed.  In  a  final  part  of  the 
evaluation,  information  is  gathered  on  the  need  to  conduct  and  the 
integration  of  a  sensitivity  and/or  uncertainty  analysis  in  the  tool. 

2.2.4.  Application  of  the  tools 

In  the  fourth  level  of  the  evaluation  framework  the  purpose  is 
to  focus  on  the  application  of  the  tools.  For  that  purpose,  various 
aspects  can  be  taken  in  consideration  such  as  the  overall  pur¬ 
pose  of  the  tool  (communication,  decision-support  or  learning),  the 
user  category  (companies,  government/authorities,  NGOs)  or  the 
object  of  analysis  (product,  function  or  economy)  [123].  A  study 
conducted  by  Ness  et  al.  [15]  performs  a  categorisation  of  assess¬ 
ment  tools  based  on  the  temporal  focus  of  the  tool  along  with  the 
object  of  focus  of  the  tool.  The  temporal  focus  can  be  either  retro¬ 
spective  (indicators/indices),  prospective  (integrated  assessment) 
or  both  (product-related  assessment).  The  object  of  focus  of  the 
tools  is  either  spatial,  referring  to  a  proposed  change  in  policy  (indi¬ 
cator/indices  and  integrated  assessment),  or  at  the  product  level 
(product-related  assessment).  For  this  paper,  the  same  exercise  was 
performed  as  well  as  an  evaluation  based  on  the  development  stage 
of  the  tool  and  the  frequency  of  application. 

23.  Step  2:  evaluation  of  existing  assessment  tools 

In  the  second  step  of  the  study,  the  newly  constructed  evaluation 
framework  is  used  to  evaluate  the  six  selected  assessment  tools. 

In  the  first  two  evaluation  levels  (sustainability  issues  and  tool 
attributes),  the  research  was  carried  out  with  the  use  of  a  Delphi 
panel  of  experts.  The  Delphi  method  is  a  popular  technique  for  fore¬ 
casting  and  an  aid  in  decision-making  based  on  the  opinions  of 
experts,  which  has  been  in  existence  for  over  half  a  century  [127]. 
The  aim  of  the  study  was  to  gather  a  group  of  scientist  with  exper¬ 
tise  in  the  sustainability  assessment  field,  including  at  least  one 
expert  for  each  of  the  selected  assessment  tools.  Unfortunately,  no 
experts  could  be  included  for  exergy  assessment.  As  a  consequence, 
the  Exergy  Analysis  tool  was  only  evaluated  in  the  third  and  fourth 
level  of  the  evaluation  framework,  for  which  no  Delphi  expertise 
was  required.  The  experts  first  received  documentation  about  the 
aim  and  planning  of  the  study  by  e-mail.  Next,  an  interactive  meet¬ 
ing  was  organised,  during  which  grades  were  assigned  to  each  of 
the  tools  for  each  of  the  sustainability  issues  and  tool  attributes 
(level  1  and  2  of  the  evaluation  framework).  Experts  were  able  to 
jointly  discuss  and  exchange  thoughts  during  the  assignment.  The 
grades  were  ordinal  numbers  from  one  to  five  by  which  one  stands 
for  the  lowest  score  and  five  for  the  highest. 

The  material  used  to  evaluate  the  assessment  tools  in  level  3  and 
4  of  the  evaluation  framework  (overall  model  structure  and  tool 
application)  consisted  of  scientific  papers  describing  the  method¬ 
ology  of  the  tools,  literature  related  to  specific  application  of  each 
of  the  assessment  tools  in  the  field,  and  practical  examples. 

2.4.  Step  3:  overview  of  strengths  and  weaknesses 

The  evaluation  of  the  first  and  second  level  of  the  evaluation 
framework,  carried  out  by  the  Delphi  panel  of  experts,  resulted  in 
two  data  matrices  consisting  of  ordinal  scores,  giving  an  overview 
of  the  strengths  and  weaknesses  of  the  different  tools.  In  order 
to  detect  significant  differences  between  tools,  the  data  matrices 
were  statistically  analysed.  Because  of  the  ordinal  character  of  the 
data,  a  Kruskal-Wallis  analysis  of  variance  was  performed.  The 
Kruskal-Wallis  test  is  a  non-parametric  method  for  testing  equal¬ 
ity  of  population  medians  amongst  groups.  However,  in  this  study 
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we  had  only  one  observation  for  every  evaluator  variable  sepa¬ 
rately,  so  no  medians  could  be  generated.  Therefore,  significant 
differences  (a  =  0.05 )  between  tools  were  evaluated  at  the  subgroup 
level.  In  the  data  matrix  of  the  sustainability  issues,  the  subgroups 
are  the  environmental,  social,  economic  and  institutional  dimen¬ 
sion.  In  the  data  matrix  of  the  tool  attributes,  the  subgroups  are 
data,  procedure,  results  and  use. 

The  data  gathered  about  the  overall  model  structure  in  the  third 
level  of  the  evaluation  framework  are  reproduced  in  the  form  of  tool 
profiles,  which  enables  rapid  comparison  of  the  different  assess¬ 
ment  tools. 

3.  Results  and  discussion 

In  comparing  sustainability  assessment  tools,  many  different 
tool  characteristics  and  aspects  are  involved.  Therefore,  an  eval¬ 
uation  framework  was  constructed,  distinguishing  four  levels  of 
evaluation:  issues  of  sustainability,  tool  attributes,  overall  model 
structure  and  tool  application,  in  which  tools  were  evaluated.  While 
recognising  the  importance  of  such  a  comparative  study,  it  is  impor¬ 
tant  to  note  that  the  results  are  not  always  very  straightforward 
because  of  many  reasons.  Although  the  Delphi  method  is  consid¬ 
ered  as  a  very  valuable  way  to  provide  information,  it  cannot  be 
avoided  that  expert  valuations  are  prone  to  some  degree  of  subjec¬ 
tivity.  Furthermore,  most  of  the  considered  tools  are  flexible,  are 
developing  over  time,  and  not  always  well  defined.  Available  liter¬ 
ature  might  be  outdated  or  not  complete.  At  last,  tools  can  be  devel¬ 
oped  for  many  different  reasons,  and  therefore  do  not  always  serve 


for  a  straightforward  comparison.  It  is  important  to  keep  these 
issues  in  mind  while  examining  the  results  of  this  study.  In  the  fol¬ 
lowing  paragraphs,  results  are  discussed  for  each  evaluation  level. 

3. 1 .  Issues  of  sustainability 

As  sustainability  is  a  complex  and  multidimensional  sub¬ 
ject,  integrated  sustainability  assessment  requires  a  multicriterial 
approach.  For  this  study,  important  sustainability  issues  were  listed 
for  the  bioenergy  field  and  tools  were  valuated  with  the  use  of  a 
Delphi  panel  of  experts.  The  lowest  score  of  1  was  given  in  case  the 
tool  did  not  address  the  aspect  at  all;  a  value  of  2  when  the  aspect 
was  only  addressed  to  a  very  limited  extend;  a  value  of  3  when  the 
tool  addresses  the  aspect  with  a  certain  methodology  but  only  par¬ 
tially;  a  value  of  4  in  case  the  aspect  is  very  well  addressed,  but  still 
could  be  improved  significantly;  and  a  value  of  5  in  case  the  aspect 
involved  is  fully  covered  (Table  3). 

The  results  give  a  clear  overview  of  the  sustainability  areas  that 
are  covered  by  the  different  tools.  It  can  be  noticed  that  most 
of  the  tools  are  environmentally  focused.  Except  for  a  few  minor 
issues,  LCA,  EIA  and  SPA  only  take  the  environmental  dimension 
into  account,  whereas  CBA  is  rather  an  economic  tool  incorporat¬ 
ing  some  environmental  issues.  For  the  environmental  dimension, 
there  is  a  broad  agreement  in  the  scientific  community  that  LCA 
is  one  of  the  strongest  players  in  the  assessment  field  [70,83], 
although  C&I  and  EIA  also  show  good  performances.  The  scope  and 
methodologies  of  EIA  have  evolved  greatly  over  the  past  decade. 
Advances  include  consideration  of  biodiversity  and  climate  change 


Table  3 

Delphi  panel  scores  (lowest  score  is  1,  highest  score  is  5)  for  the  extent  to  which  sustainability  issues  are  addressed  within  sustainability  assessment  tools  for  bioenergy 
systems. 


Sustainability  issues 

C&I 

LCA 

EIA 

CBA 

SPA 

Environmental  dimension 

Air  quality 

2 

5 

4 

4 

2 

Soil  quality 

4 

2 

4 

1 

1 

Land  use 

3 

3 

3 

3 

5 

Water  quality 

4 

3 

4 

1 

1 

Water  use 

4 

5 

4 

3 

1 

Biodiversity 

3 

3 

3 

1 

1 

Deforestation 

4 

3 

3 

1 

3 

Desertification 

3 

3 

3 

1 

1 

Landscape 

2 

4 

4 

2 

1 

Agrochemicals 

3 

3 

3 

1 

3 

Waste 

4 

4 

3 

1 

1 

Energy  balance 

4 

5 

3 

5 

5 

Fossil  resources 

3 

5 

3 

4 

5 

Social  dimension 

Labour  conditions 

3 

1 

1 

2 

1 

Human  safety  &  health 

3 

1 

3 

2 

1 

Human  rights 

3 

1 

1 

1 

1 

Access  to  resources 

3 

1 

1 

1 

1 

Food  &  energy  supply 

2 

1 

1 

1 

2 

Capacity  building 

3 

1 

1 

1 

1 

Democratic  participation 

3 

1 

2 

1 

1 

Property  rights 

3 

1 

1 

1 

1 

Fair  trade 

3 

1 

1 

1 

1 

Social  acceptance 

3 

1 

3 

1 

1 

Economic  dimension 

Viability 

3 

1 

1 

4 

1 

Long  term  perspective 

3 

3 

1 

3 

2 

Reliability  of  resources 

2 

3 

1 

2 

2 

Reliability  of  technology 

1 

1 

1 

3 

2 

Risk  minimisation 

2 

1 

1 

1 

1 

Local  economy 

3 

1 

1 

1 

1 

Reliability  of  energy 

2 

1 

1 

3 

3 

Other  desirable  developments. 

3 

1 

1 

1 

1 

Institutional  dimension 

Policy  and  legislation 

5 

1 

5 

1 

2 

Institutional  capacity 

2 

1 

2 

1 

1 

Institutional  well-being 

3 

1 

1 

1 

1 

Environmental  management 

3 

1 

2 

1 

1 
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issues  [24].  Of  the  six  selected  tools,  social  issues  are  only  addressed 
by  C&I.  The  reason  behind  this  is  probably  the  difficulty  with 
which  social  issues  of  sustainability  are  translated  into  measur¬ 
able  quantities,  or  at  least  into  operational  terms.  According  to 
Becker  [1]  approaches  to  quantification  and  operationalisation  of 
social  dimensions  must  be  carefully  restricted  to  those  issues  that 
can  be  described  meaningfully  by  numerical  or  analytical  tools  and 
methods.  Yet,  C&I  cover  most  of  the  issues  of  social  sustainabil¬ 
ity  although  some  sustainability  issues  such  as  labour  conditions, 
human  safety  and  property  rights  cannot  get  a  better  indicator  than 
a  rank  or  go/no  go.  Only  for  competition  with  food,  no  appropriate 
indicator  was  found  yet  in  C&I.  The  institutional  dimension  is  very 
poorly  addressed  in  all  of  the  tools  except  for  the  legislative  aspect, 
which  is  of  significant  importance  in  EIA  and  C&I. 

The  Kruskal-Wallis  analysis  that  was  performed  on  the  results 
of  the  sustainability  issues  for  each  of  the  sustainability  sublevels 
(environmental,  social,  economic  and  institutional  dimension) 
reinforced  the  visual  conclusions  by  not  revealing  any  significant 
differences  between  the  tools  except  for  the  social  dimension  ip- 
values  for  the  environmental,  social,  economic  and  institutional 
dimension  are  0.014,  0,  0.011  and  0.009,  respectively). 

To  find  out  which  sustainability  issues  are  addressed  in  EA,  sci¬ 
entific  literature  was  consulted.  The  relationship  between  exergy 
and  sustainability  is  well  explained  in  studies  of  Cornelissen  [128], 
Rosen  and  Dincer  [106],  Dincer  and  Rosen  [129]  and  Dewulf  et  al. 
[130].  It  seems  that  the  focus  of  EA  is  limited  to  the  environmental 
dimension,  in  particular  emissions  and  natural  resource  use.  The 
authors  state  that  environmental  effects  associated  with  emissions 
and  resource  depletion  can  be  expressed  in  terms  of  one  exergy- 
based  indicator.  In  contrast  with  energy,  which  can  never  be  lost  as 
it  is  conserved  according  to  the  first  law  of  thermodynamics,  exergy 
can  be  lost  due  to  internal  irreversibilities.  Therefore,  sustainabil¬ 
ity  is  seen  to  increase  and  environmental  impact  to  decrease  as  the 
exergy  efficiency  of  a  process  increases  [106].  Some  studies  were 
found  relating  EA  and  economics  by  determining  the  appropriate 
allocation  of  economic  resources  so  as  to  optimize  the  design  and 
operation  of  a  system,  or  by  determining  the  economic  feasibility 
and  profitability  of  a  system.  Examples  of  exergy-based  economic 
analysis  methods  are  thermoeconomics,  second-law  costing,  cost 
accounting  and  exergoeconomics.  Dewulf  et  al.  [130]  give  a  concise 
overview  of  the  application  of  EA  in  the  combined  environmental 
and  economic  field,  and  debate  the  actual  merits  of  such  an  exergy- 
based  economic  approach  [  1 01  ].  No  exergy  studies  were  found  that 
address  social  issues  of  sustainability. 

In  general,  the  major  problems  in  sustainability  assessment  are 
data  availability  and  methodological  agreement.  For  some  sus¬ 
tainability  issues,  such  as  energy  balance,  methodologies  are  well 
defined  and  accepted,  but  for  others,  methodologies  are  heavily 
debated.  For  example,  the  ideas  on  how  to  inventory  and  charac¬ 
terize  land  use  are  incomplete  and  quite  diverse  [131],  not  only 
between  tools  but  also  within  tools.  In-depth  discussions  on  how 
to  integrate  land  use  within  LCA  are  still  going  on  and  can  be  found 
in  [85,132-136]  midst  others.  The  same  difficulties  are  faced  for 
assessing  biodiversity.  Although  a  few  indicators  related  to  biodi¬ 
versity  already  exist  (e.g.  species  diversity,  existence  of  threatened 
and  protected  species),  there  is  no  agreement  on  a  common  denom¬ 
inator  or  calculation  procedure  within  sustainability  assessment  of 
bioenergy  systems.  At  date,  the  problem  is  mostly  tackled  by  deter¬ 
mining  protected  areas  where  intervention  is  not  allowed  to  take 
place.  The  reason  why  in  most  cost-benefit  analyses  some  of  the 
environmental  and  most  of  the  social  costs  are  omitted,  is  often 
because  they  are  too  difficult  to  estimate  from  an  economic  point  of 
view  and  to  translate  into  monetary  terms.  For  the  use  of  agrochem¬ 
icals,  for  instance,  cost-benefit  analyses  may  consider  the  costs  of 
treating  wastewater  from  the  chemical  plant  where  the  agrochem¬ 
icals  are  produced,  but  the  effect  on  nature  is  often  omitted  because 


it  seems  too  complicated  to  determine  the  values  for  the  deterio¬ 
rated  nature.  It  can  be  concluded  that  within  existing  assessment 
tools  the  same  sustainability  issues  are  often  left  out  of  considera¬ 
tion  due  to  a  lack  of  possible  calculation  methodologies  or  available 
data.  Future  research  should  focus  on  those  sustainability  issues  for 
which  no  common  calculation  methodology  could  be  defined  yet. 

3.2.  Tool  attributes 

In  the  second  evaluation  step,  tools  were  evaluated  by  the  Del¬ 
phi  panel  of  experts  on  basis  of  their  tool  characteristics  regarding 
use  of  data,  followed  procedure,  reproduction  of  results,  and  use  of 
the  tool.  Delphi  scores  ranged  from  1  to  5,  corresponding  with  very 
poor,  poor,  moderate,  good  or  very  good.  The  resulting  data  matrix 
can  be  found  in  Table  4.  Some  of  the  tool  attributes  are  also  of  impor¬ 
tance  in  the  next  level  of  comparison  (overall  model  structure)  and 
will  be  discussed  in  more  detail  there. 

At  first  glance,  no  general  differences  can  be  observed 
between  the  tools.  This  perception  is  confirmed  by  the  statisti¬ 
cal  Kruskal-Wallis  analysis,  which  does  not  detect  any  significant 
differences  between  the  different  tools  (p-values  for  data,  proce¬ 
dure,  results  and  use  are  0.453,  0.220,  0.533,  0.192,  respectively). 
Although  there  might  not  be  a  statistical  significant  difference  in 
general,  many  smaller  differences  can  be  observed  on  the  various 
aspects  in  particular. 

Although  input  data  is  generally  a  difficult  issue  in  sustainability 
assessment,  LCA  shows  many  advantages.  Extensive  high-quality 
data  sets  have  been  gathered  in  the  past  decennium  and  are  reg¬ 
ularly  updated.  CBA  scores  well  because  of  advantages  related  to 
valuing  such  as  a  similar  unit  for  all  impacts  which  facilitates  inte¬ 
gration.  Concerning  data  availability,  need  and  integration,  EIA 
shows  difficulties.  The  tool  is  very  site-specific  and  detailed,  which 
requires  extensive  application  of  data  and  hampers  integration. 
Low  scores  are  also  assigned  to  SPA,  which  has  the  disadvantage 
of  being  a  very  recent  and  not  yet  fully  developed  assessment  tool. 
Only  few  sustainability  issues  are  considered  and  clear  assessment 
guidelines  are  still  lacking.  This  might  change  in  the  near  future. 

On  the  aspects  of  procedure,  C&I  receives  the  best  overall 
score.  Indicators  are  simple  and  flexible  measures,  most  often 
quantitative,  that  represent  a  state  of  economic,  social  and/or  envi¬ 
ronmental  development  in  a  defined  region.  When  selected  with 
great  care,  C&I  can  be  a  strong  measure  for  sustainability  assess¬ 
ment.  But  in  the  case  of  poorly  chosen  indicators  a  variety  of 
problems  occur,  amongst  which  overaggregation,  measuring  irrel¬ 
evant  aspects,  dependence  on  a  false  model,  deliberate  falsification, 
and  incompleteness  [137].  Other  advantages  of  C&I  are  an  almost 
full  coverage  of  the  sustainability  field,  whereas  other  tools  rather 
focus  on  one  or  two  sustainability  dimensions,  and  the  ability  to  use 
static  as  well  as  dynamic  data,  whereas  most  other  tools,  except  for 
EIA  use  only  static  data.  A  positive  aspect  to  note,  is  that  all  tools 
score  well  on  transparency  and  not  any  is  prone  to  high  subjectivity. 

With  regard  to  the  results,  all  tools  generally  show  high  scores. 
Results  are  in  most  tools  rather  compact  and  clear,  easy  to  under¬ 
stand  without  expert  knowledge.  Only  for  SPA  one  weakness  can  be 
observed  for  applying  a  complex  calculation  method  which  makes 
it  difficult  to  interpret  the  results. 

Finally,  with  respect  to  the  use  of  the  tool,  all  of  them  show 
the  advantage  of  being  able  to  integrate  stakeholder  input  into  the 
modelling,  whereas  accessibility  forms  a  bottleneck  for  each  of  the 
tools.  This  means  that  expert  knowledge  is  needed  to  apply  the 
tool,  particularly  in  the  case  of  EIA  and  SPA.  LCA  scores  best  on 
time  consumption  and  assistance  considering  the  rather  limited 
time  that  is  needed  for  the  assessment  and  the  good  guidelines  and 
assistance  that  are  available,  the  latter  being  the  opposite  for  SPA. 
EIA  seems  to  be  worst  on  these  two  issues,  as  well  when  handiness 
is  concerned. 
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Table  4 

Delphi  panel  scores  (lowest  score  is  1,  highest  score  is  5)  for  tool  attributes  of  sustainability  assessment  tools  for  bioenergy  systems. 


Tool  attribute 

C&I 

LCA 

EIA 

CBA 

SPA 

Data 

Availability 

3 

4 

2 

3 

2 

Need 

3 

3 

2 

3 

2 

Type 

3 

5 

5 

5 

5 

Integration 

4 

1 

1 

4 

1 

Correctness 

3 

4 

3 

3 

3 

Procedure 

Flexibility 

5 

1 

3 

3 

3 

Coverage 

4 

1 

1 

4 

2 

Objectivity 

3 

3 

4 

3 

3 

Transparency 

4 

5 

4 

4 

4 

Validation 

4 

4 

2 

1 

3 

Time  scale 

4 

1 

1 

5 

2 

State-of-the-art 

5 

5 

5 

3 

2 

Results 

Univocity 

5 

4 

4 

3 

4 

Simplicity 

5 

4 

4 

3 

2 

Type 

3 

5 

4 

4 

5 

Clarity 

4 

2 

3 

4 

3 

Communication 

4 

5 

3 

4 

3 

Use 

Flexibility 

5 

5 

5 

4 

5 

Accessibility 

4 

3 

2 

3 

2 

Application  cost 

3 

3 

3 

3 

3 

Application  field 

5 

3 

4 

2 

4 

Time  consumption 

3 

4 

2 

3 

3 

Handiness 

4 

3 

2 

3 

3 

Assistance 

3 

5 

2 

3 

2 

3.3.  Overall  model  structure 

This  part  of  the  evaluation  is  based  on  literature  review  and  the 
results  can  be  found  in  the  form  of  tool  profiles  in  Appendix  A.  A 
discussion  of  the  results  follows  here. 

The  tool  description  specifies  in  short  what  the  overall  purpose 
is  of  using  the  tool  and  which  aspects  are  assessed.  Although  some 
tools  are  rather  used  for  communication  to  provide  others  with 
information  (C&I),  whereas  other  tools  serve  as  decision-support 
for  operative  or  strategic  decision  making  (CBA,  EIA)  or  are  used 
by  companies  for  internal  learning  purposes  (EA),  or  for  all  these 
purposes  together  (LCA,  SPA),  they  all  contribute  in  some  way  to 
sustainability  assessment  and  can  to  some  extend  be  used  for  the 
assessment  of  bioenergy  systems. 

The  tool  concept  gives  an  overview  of  the  different  methodolog¬ 
ical  steps  that  must  be  executed  while  applying  the  tool  and  is 
related  to  the  concept  focus  of  the  tool.  A  tool  typically  consists  of 
a  systematic  data  and/or  information  gathering  procedure  and  will 
often  also  include  a  calculation  algorithm  [123].  Tools  that  empha¬ 
sise  procedure  are  C&I  and  EIA,  whereas  the  others  tools  (LCA, 
CBA,  SPA  and  EA)  rather  focus  on  the  computational  algorithm.  For 
example,  the  EIA  methodology  gives  descriptions  of  an  information 
gathering  procedure  (in  order  to  make  decision)  in  which  various 
algorithms  may  be  used,  whereas  the  LCA  methodology  defines  the 
type  of  computational  model  to  be  used  (a  computational  model  of 
material  and  energy  flows  related  to  a  functional  unit).  Since  pro¬ 
cedure  aims  at  describing  the  best  way  to  reach  a  decision,  while 
algorithms  aim  at  finding  the  better  solution,  an  analytical  tool 
might  be  used  as  part  of  a  procedural  tool  [125].  As  a  consequence 
of  their  focus  on  algorithm,  analytical  tools  show  the  disadvantage 
that  sustainability  issues  that  cannot  be  modelled  appropriately 
can  hardly  be  integrated  in  the  assessment.  This  again  is  related  to 
the  rather  quantitative  or  qualitative  character  of  the  tool  and  will 
be  discussed  further  on. 

The  object  of  analysis  identifies  the  focus  of  decision.  If  the 
object  of  analysis  is  a  project,  such  as  a  bioenergy  implementa¬ 
tion  project,  EIA  or  CBA  is  the  natural  choice  rather  than  LCA  or 
another  method  developed  for  another  object,  which  could  be  pre¬ 


ferred  when  assessing  for  example  1  MJ  of  biofuel.  C&I  as  well  as 
SPA  and  EA  could  lay  their  focus  on  any  kind  of  object  or  system, 
be  it  a  product,  process  or  project. 

Assessment  tools  usually  involve  comparison  in  various  ways. 
The  basis  for  comparison  is  what  is  being  kept  constant  in  a  com¬ 
parison  [123]  and  depends  in  most  cases  on  the  object  of  analysis. 
For  example,  in  LCA,  all  analyses  are  relative  to  the  functional  unit, 
which  measures  the  unit  good  or  service  that  is  being  delivered.  In 
case  of  bioenergy  this  would  be  one  unit  of  the  fossil  energy  ref¬ 
erence.  The  same  applies  for  SPA  and  EA.  C&I  on  the  other  hand  is 
usually  not  used  to  make  a  comparison  between  different  products 
or  projects,  but  rather  to  make  an  evaluation  of  the  same  project 
over  time.  EIA  does  not  necessarily  require  a  reference  object  either, 
however,  in  most  EIA  studies,  various  alternative  projects  are  com¬ 
pared  amongst  which  a  zero  alternative  [89].  This  is  also  the  case 
for  CBA. 

The  degree  of  standardisation  or  harmonisation  is  an  important 
factor  because  even  small  differences  in  methodological  factors  can 
influence  the  results.  Of  all  the  tools  selected  for  this  study,  LCA 
is  the  only  tool  that  is  standardised  by  the  ISO.  The  tool  evolved 
over  the  last  three  decades  from  a  relatively  vague  framework 
for  conducting  assessments,  into  a  rigorous  set  of  internation¬ 
ally  standardised  ISO  guidelines  [71,72,123,138].  However,  Dewulf 
and  Van  Langenhove  [109]  state  that  some  further  methodological 
developments  are  still  needed  for  the  LCIA  models  and  methods. 
For  C&I,  internationally  harmonised  frameworks  and/or  guidelines 
exist  for  specific  systems,  often  as  part  of  certification  systems. 
Also  for  bioenergy,  different  C&I  frameworks  have  been  developed 
[52-54,121].  According  to  Moberg  [95]  an  international  agree¬ 
ment  on  a  general  procedure  exists  for  EIA  and  because  of  legal 
implementation,  many  countries  defined  their  own  harmonised 
procedure.  Also  for  CBA,  international  guidelines  exist.  For  exam¬ 
ple,  the  European  Union  established  an  extensive  guide  for  CBA  of 
different  kinds  of  investment  projects  [139].  On  the  methodology 
of  SPA,  little  information  is  available  because  of  very  recent  devel¬ 
opment  of  the  tool.  For  EA,  no  standardisation  or  harmonisation 
has  been  performed  but  quite  some  information  is  encountered 
regarding  the  methodology  [104,105,107,110,140]. 
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Similarity  can  be  found  between  the  degree  of  standardisa¬ 
tion  or  harmonisation  and  the  frequency  of  use.  With  respect  to 
sustainability  assessment  of  bioenergy  systems,  LCA  is  one  of  the 
most  well-known  and  applied  tools,  together  with  C&I  within  cer¬ 
tification  systems.  CBA,  in  its  turn,  seems  to  be  the  most  well 
established  and  utilised  tool  in  economic  decision  making  [95].  EIA 
is  a  well-known  and  frequently  used  tool  for  site-specific  invest¬ 
ment  projects  where  participation  of  all  the  stakeholders  involved 
is  of  great  importance.  To  date,  SPA  is  only  used  by  its  developers. 
And  although  many  scientists  suggest  that  the  impact  of  energy 
resource  utilization  on  the  environment  and  the  achievement  of 
increased  resource-utilization  efficiency  are  best  addressed  by  con¬ 
sidering  exergy  [105,1 13,141  ],  the  EAtool  still  is  not  very  frequently 
applied  in  a  non-scientist  environment,  perhaps  due  to  the  unfa¬ 
miliarity  with  the  exergy  concept. 

With  regard  to  the  time  perspective ,  C&I  is  generally  seen  as  a 
retrospective  tool,  whereas  CBA,  EIA  and  SPA  have  mainly  prospec¬ 
tive  purposes,  and  LCA  and  EA  can  be  considered  as  both  retro- 
and  prospective.  Concerning  the  temporal  and  spatial  scale,  these 
aspects  are  in  principle  defined  by  the  temporal  and  spatial  dis¬ 
tribution  of  the  assessed  impacts  of  the  system  under  evaluation. 
Therefore,  sustainability  indicators  within  C&I  can  be  divided 
roughly  into  state  and  trend  indicators  and  have  specific  spatial 
scales  and  time  horizons.  Becker  [1]  presents  a  clear  overview 
of  sustainability  indicators  in  a  space/time  frame.  In  agriculture, 
spatial  scales  range  from  plot-level  and  small-scale  gathering  to 
agribusiness  and  large  plantations,  and  temporal  scales  range  from 
short-season  crops  to  perennials  and  long  rotations  in  swidden 
agriculture  [142].  While  conventional  environmental  assessment 
techniques  such  as  EIA  focus  only  on  either  manufacturing  pro¬ 
cesses  or  end-of-life  disposal  (or  reuse),  LCA  considers  the  life  cycle 
of  a  system,  or  the  entire  chain  of  events  and  activities  that  are  nec¬ 
essary  to  support  the  product  or  process  [71,138,143].  This  is  often 
called  the  cradle-to-grave  approach,  and  has  the  obvious  advan¬ 
tage  of  revealing  potentially  significant  but  hidden  environmental 
impacts.  Instead  of  focussing  attention  on  large,  concentrated  and 
readily  apparent  point  sources  of  impacts  -  for  example,  a  manu¬ 
facturing  plant  -  LCA  also  takes  into  account  dispersed  activities 
whose  cumulative  effects  may  prove  to  be  critical  as  well  [138]. 
In  many  cases,  EA  takes  on  a  life-cycle  perspective  as  well,  quan¬ 
tifying  the  cumulated  exergy  consumption  of  a  product  or  process 
from  “cradle  to  grave”.  In  this  regard,  it  is  similar  to  LCA.  Moreover, 
according  to  Dewulf  et  al.  [101]  EA  can  be  part  of  an  LCA,  repre¬ 
senting  a  method  for  the  life-cycle  impact  assessment  of  resource 
consumption.  A  coupling  of  Exergy  Analysis  and  life  cycle  inven¬ 
tory  databases  (e.g.  Ecoinvent),  has  been  put  into  practice  by  the 
authors.  The  developers  of  SPA  followed  another  train  of  thought 
considering  geographical  system  balances  of  resources  and  related 
effects  rather  than  a  life-cycle  approach,  what  makes  the  assess¬ 
ment  a  snapshot  in  time  rather  than  an  integration  of  impacts  that 
occur  over  time,  and  no  future  developments  are  taken  into  con¬ 
sideration.  A  cost-benefit  analysis  can  give  useful  results  for  the 
present  costs  and  benefits,  but  future  benefits  are  strongly  depen¬ 
dent  on  the  interest  rate  applied  in  the  analysis.  Jorgensen  [103] 
state  that  the  use  of  depreciation  of  future  benefits  due  to  introduc¬ 
tion  of  an  interest  rate  will  inevitably  emphasize  too  much  present 
benefits.  In  addition,  predictions  on  future  interest  rate  have  a  very 
high  uncertainty. 

The  spatial  scale  is  related  to  the  spatial  boundaries  and  the  spa¬ 
tial  horizon  of  the  analysed  system.  Whereas  most  of  the  selected 
tools  can  be  applied  on  different  spatial  scales,  EIA  is  a  particu¬ 
larly  site-specific  tool  [125].  Although  all  the  tools  make  use  of 
environmental  data,  the  environmental  data  of  EIA  are  usually 
more  detailed  and  site  or  project  specific.  For  example,  EIA  can 
often  involve  taking  into  account  the  duration  and  concentration 
of  emitted  pollutants  and  the  assessment  of  actual  impacts  on  the 


environment  [123],  whereas  the  other  tools  keep  it  less  situation- 
specific  to  serve  as  rather  generic  tools.  The  site-specificity  of  EIA 
makes  it  a  time  and  resource  demanding  tool.  On  the  opposite  side, 
indicators,  and  in  particular  social  indicators,  can  be  used  to  refer  to 
all  levels  of  the  spatial  hierarchy,  with  varying  degrees  of  relevance. 
Yet,  in  different  societies  with  varying  cultural  and  legal  practice, 
assessment  of  social  indicators  will  always  be  a  complex  issue  that 
cannot  easily  be  put  into  operation  [1  ]. 

There  is  no  straightforward  answer  whether  an  assessment  tool 
should  be  purely  quantitative  or  both  quantitative  and  qualitative. 
Not  all  sustainability  issues  can  easily  be  quantified.  The  accuracy 
of  the  outcome  strongly  depends  on  the  accuracy  of  the  measure¬ 
ments  and  the  modelling  within  the  tool.  Besides,  the  combination 
of  qualitative  and  quantitative  results  impedes  aggregation  into  one 
single  outcome.  Therefore,  in  some  tools  only  well-defined  quan¬ 
titative  parameters  are  counted  in.  On  the  other  hand,  in  order  to 
fully  address  sustainability,  all  issues  should  be  considered,  also  the 
non-quantifiable  ones.  Of  all  the  tools  regarded  in  this  study,  only 
C&I  and  EIA  have  both  a  quantitative  and  qualitative  character.  The 
other  tools  are  purely  quantitative. 

The  quantitative  and/or  qualitative  character  of  a  tool  also  influ¬ 
ences  the  way  in  which  the  results  are  reproduced.  According  to 
Becker  [1  ],  three  basic  approaches  can  be  distinguished:  unranked 
lists  of  heterogeneous  indicators,  scoring  systems  with  unified 
dimensions,  and  system  descriptors.  C&I,  LCA,  EIA  and  SPA  belong 
to  this  first  group,  although  results  might  be  partially  aggregated  for 
example  into  impact  categories  as  is  the  case  in  LCA.  The  advantage 
of  listing  results  as  heterogeneous  indicators  gives  the  advantage 
of  transparency,  but  the  disadvantage  of  complex  interpretation. 
Because  of  their  ability  to  transform  all  parameters  into  one  unit, 
CBA  and  EA  are  tools  that  belong  to  the  second  group  of  scoring  sys¬ 
tems  with  unified  dimensions.  Such  tools  have  the  advantage  that 
they  permit  comparison  of  and  calculation  with  different  quantities 
in  a  uniform  dimension.  However,  according  to  Becker  this  assumes 
a  congruent  basis  of  calculation,  which  in  reality  differs  consider¬ 
ably  depending  on  the  underlying  value  judgement.  Furthermore, 
in  case  of  CBA  monetary  values  are  not  sufficiently  consistent  with 
ecosystem  structure  and  function,  and  therefore  their  aggregation 
may  lead  to  inadequate  environmental  decisions. 

The  assignment  of  weight  scores  is  considered  as  a  very  contro¬ 
versial  step  mainly  because  it  involves  social  political  and  ethical 
value  choices  [109].  Therefore,  weighing  of  impacts  is  in  most  tools 
not  integrated,  or  indicated  as  an  optional  step.  For  example,  in 
the  inventory  phase  (LCIA)  of  LCA,  a  number  of  methods  is  pro¬ 
posed  to  conduct  weighing.  These  methods  are  divided  in  two 
general  groups:  (1)  problem-oriented  approaches  and  (2)  damage- 
oriented  methods.  Examples  of  the  problem-oriented  approaches 
are  the  CML-method  and  EDIP  method,  and  examples  of  damage- 
oriented  methods  are  EPS  2000  and  Eco-Indicator  99.  The  IMPACT 
2002+  method  attempts  to  link  the  problem-  and  damage-oriented 
approaches  in  a  common  framework  [109]. 

An  optimal  application  of  a  model  or  tool  requires  the  execution 
of  a  sensitivity  and  uncertainty  analysis  be  fore  the  final  conclusions 
and  recommendations  of  the  study  are  made.  However,  because 
of  time  and/or  resource  restriction,  this  part  is  often  omitted.  Of 
the  six  tools  considered  in  this  study,  only  two  of  them,  LCA  and 
CBA,  include  a  sensitivity  analysis  and  to  a  limited  extend  also 
an  uncertainty  analysis  in  their  methodology,  however  optional. 
Only  in  a  few  case  studies  sensitivity  and/or  uncertainty  analyses 
were  conducted  although  it  was  often  stated  that  the  methodol¬ 
ogy  applied  was  prone  to  considerable  uncertainties.  With  regard 
to  assessment  tools  in  general,  three  kinds  of  uncertainty  can  be 
distinguished:  data  uncertainty,  model  uncertainty  and  scenario 
uncertainty.  Small-scale  models  are  less  prone  to  uncertainty  than 
large-scale  models,  because  the  latter  cover  in  most  cases  longer 
time  spans,  and  therefore  need  to  make  more  probability  assump- 
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Fig.  2.  Possible  field  of  application  of  sustainability  assessment  tools  based  on  their 
temporal  focus,  along  with  the  object  of  focus  of  the  tool. 

Adapted  from  Ness  et  al.  [15]. 

tions  [  1  ].  More  specific,  in  case  of  C&I  uncertainties  can  be  caused  by 
the  diversity  of  the  set  of  criteria  and  indicators,  criteria  content, 
indicator  quality,  lack  of  validity,  poor  data  sets  amongst  others 
[144].  In  life-cycle  assessment,  various  aspects  such  as  input  param¬ 
eter  values,  system  boundaries,  allocation  procedure,  and  fossil 
reference  system,  give  rise  to  wide  ranges  of  results  [83].  In  a  cost- 
benefit  analysis  the  accuracy  of  the  outcome  highly  depends  on 
which  costs  are  included  in  the  analysis  and  how  accurately  costs 
and  benefits  have  been  estimated.  Various  peer-reviewed  studies 
[145,146]  show  that  differences  between  actual  costs  and  bene¬ 
fits  and  estimated  costs  and  benefits  can  be  up  till  more  than  fifty 
percent  and  that  the  outcomes  of  cost-benefit  analyses  should  be 
treated  with  caution  because  they  may  be  highly  inaccurate.  Fur¬ 
thermore,  Flyvbjerg  et  al.  [147]  state  that  inaccurate  cost-benefit 
analyses  may  be  argued  to  be  a  substantial  risk  in  planning,  because 
inaccuracies  of  the  size  documented  are  likely  to  lead  to  ineffi¬ 
cient  decisions.  EIA  is  prone  to  similar  uncertainties  as  in  C&I  in 
the  phases  in  which  indicators  are  used  and  to  some  extend  to  the 
same  uncertainties  of  CBA  in  case  a  financial  analyses  is  executed. 
For  SPA,  no  information  was  found  on  the  risk  of  uncertainty  while 
applying  the  tool.  In  EA  at  last,  a  certain  degree  of  uncertainties  is 
found  in  exergy  estimations  of  products  and  processes. 

3.4.  Application  of  tools 

In  the  fourth  and  last  level  of  the  evaluation,  an  approach  was 
followed  in  which  two  factors  are  considered:  temporal  char¬ 
acteristics  and  focus  area.  As  is  discussed  earlier  in  this  paper, 
temporal  characteristics  evaluate  the  time  frame  wherein  a  product 
or  projects  has  an  impact,  as  well  as  the  time  perspective  applied 
to  the  object  analysed  which  can  be  retrospective  or  prospective. 
The  former  is  accounting/monitoring,  keeping  record  of  progress, 
which  can  be  useful  when  searching  for  preferable  changes  in  for 
example  production,  behaviour  or  policies  and  to  indicate  sustain¬ 
able  development.  Prospective  approaches,  on  the  other  hand,  are 
seeking  to  predict  future  situations  and  can  thereby  facilitate  com¬ 
parison  between  different  new  products,  plans,  etc.  [123].  The  focus 
areas  can  be  (1)  indicators  and  indices,  which  can  be  further  bro¬ 
ken  down  into  non-integrated  and  integrated,  (2)  product-related 
assessment  tools  with  the  focus  on  the  material  flow  and/or  energy 
flows  of  a  product  or  service  from  a  life  cycle  perspective,  and  (3) 
integrated  assessment,  which  are  tools  usually  focused  on  policy 
change  or  project  implementation.  Fig.  2  gives  an  overview  for  the 
six  tools  regarded  in  this  paper.  As  stated  by  Ness  et  al.  [15]  indica¬ 
tors  and  indices  (i.e.  aggregated  indicators),  which  are  continuously 
measured  and  calculated,  allow  for  the  tracking  of  longer-term 
sustainability  trends  from  a  retrospective  point  of  view.  Product- 
related  tools  such  as  LCA  and  EA  allow  both  retrospective  and 
prospective  assessments.  Integrated  assessment  tools  like  EIA,  CBA 
and  SPA  are  mostly  used  for  supporting  decisions  related  to  a  policy 
or  a  project  in  a  specific  region  and  have  an  ex-ante  focus.  Many  of 
these  tools  are  based  on  systems  analysis  approaches  and  integrate 
environmental  and  society  issues. 

Besides  the  area  of  applicability,  a  categorization  of  tools  can  also 
be  made  concerning  their  development  and  frequency  of  use.  For 


LCA,  CBA  and  EIA,  there  are  relatively  well  established,  transparent 
and  generally  agreed  guidelines  available,  whereas  for  C&I  and  for 
EA  the  practitioner  is  free  to  use  a  personal  interpretation  of  the 
general  principles.  Newer  tools  such  as  SPA  represent  an  area  where 
guideline  establishment  is  still  in  early  developmental  stages. 

3.5.  Integration  of  existing  tools 

As  can  be  concluded  from  the  previous  analysis,  the  various 
existing  sustainability  assessment  tools  are  not  necessarily  com¬ 
plementary,  but  some  of  them  may  be  well  combined  in  practice 
in  the  form  of  a  toolbox.  For  the  assessment  of  bioenergy  systems, 
impacts  must  be  considered  both  in  short  and  long  term  and  it 
involves  tracking  on-going  activities  (the  accounting  approach),  as 
well  as  choosing  between  alternative  materials/products/actions 
(the  project  approach).  The  former  is  important  for  the  generation 
of  alternatives,  the  latter  for  the  comparison  of  alternatives.  There¬ 
fore,  tools  that  focus  on  the  former  (i.e.  analytical  tools  such  as  LCA, 
CBA,  SPA  and  EA)  might  be  used  as  part  of  a  tool  that  focuses  on 
the  latter  (i.e.  procedural  tools  such  as  C&I  and  EIA).  As  the  pro¬ 
cedure  of  C&I  seems  to  be  more  clear,  transparent  and  efficient 
compared  to  this  of  EIA,  it  would  be  recommended  to  use  the  pro¬ 
cedural  framework  of  C&I  within  the  toolbox.  Nevertheless,  EIA  also 
has  some  very  useful  procedural  parts  such  as  the  screening  process 
and  mitigation  of  negative  impacts,  which  could  be  integrated  into 
the  C&I  framework.  With  regard  to  the  analytical  part  of  the  tool¬ 
box,  the  LCA  methodology  stands  out  for  its  well-developed  and 
clear  assessment  of  environmental  impacts,  although  EA  could  be  a 
decent  alternative  for  the  calculation  of  energy  efficiencies.  Where 
economic  sustainability  indicators  are  concerned,  CBA  seems  to  be 
the  most  useful  tool  to  provide  calculation  algorithms,  although 
many  issues  in  this  sustainability  dimension  for  which  no  common 
calculation  algorithm  exists  yet  will  require  considerable  additional 
research  efforts.  In  the  social  sustainability  field,  the  most  useful 
calculation  procedures  are  encountered  in  the  C&I  methodology, 
however  also  in  this  field  commonly  accepted  calculation  proce¬ 
dures  are  still  lacking  for  various  sustainability  issues. 

Even  more  interesting  would  be  the  construction  of  a  combined 
software  toolbox,  which  includes  an  application  of  geographic 
information  systems  (GIS)  and  remote  sensing  for  determining 
spatial  characteristics,  together  with  a  model  that  assesses  the  tem¬ 
poral  dimension  of  impacts,  as  well  as  a  methodology  to  conduct 
sensitivity  and  uncertainty  analysis,  and  finally  also  an  instrument 
which  enables  decision  makers  to  weight  sustainability  issues  and 
compare  alternatives  against  each  other.  It  is  clear  that,  for  such 
a  tool  to  be  created,  far  more  research  has  to  be  conducted.  Sci¬ 
entists  of  many  different  fields  of  expertise  have  to  combine  their 
knowledge  and  forces.  Solutions  must  be  found  for  those  sustain¬ 
ability  issues  for  which  no  methods  can  be  recommended  yet. 
Besides,  this  paper  represents  only  a  small  selection  of  sustainabil¬ 
ity  assessment  tools.  It  would  be  recommendable  to  extend  this 
study  to  other  assessment  methodologies  such  as  Multi-Criteria 
Analysis,  Risk  Assessment,  Strategic  Environmental  Assessment, 
Social  Impact  Assessment,  Ecological  Footprint,  Life  Cycle  Costing, 
and  many  more. 

4.  Conclusions 

The  use  of  biomass  for  energy  as  part  of  a  sustainable  devel¬ 
opment  strategy  is  heavily  debated  due  to  many  concerns  about 
sustainability.  Bioenergy  systems  are  complex  systems  having 
an  impact  on  various  levels  of  nature  and  society  and  involving 
many  stakeholders.  Furthermore,  sustainability  is  an  essentially 
contested  notion  because  it  is  inherently  complex,  normative, 
subjective  and  ambiguous  [18].  As  a  consequence,  sustainability 
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assessment  of  bioenergy  systems  is  a  highly  demanding  and  com¬ 
plicated  task.  Nevertheless,  many  researchers  have  been  engaged 
in  a  quest  to  develop  tools  and  methods  to  tackle  this  difficult  job. 
While  many  started  working  from  a  mono-disciplinary  basis,  others 
made  an  attempt  to  develop  generic  tools  and  instruments.  This  has 
resulted  in  a  diverse  field  of  approaches  [23  ].  This  paper  tried  to  give 
a  better  insight  in  the  complex  world  of  sustainability  assessment 
of  bioenergy  systems  by  analysing  and  comparing  the  applicabil¬ 
ity  and  performance  of  a  number  of  basic  sustainability  assessment 
tools  that  can  be  applied  in  this  sense.  First,  a  framework  was  con¬ 
structed  for  the  evaluation  of  the  selected  assessment  tools,  which 
takes  all  issues  into  account  that  are  of  importance  for  assessing 
bioenergy.  In  the  second  step  a  Delphi  panel  of  experts  was  con¬ 
sulted,  as  well  as  a  considerable  amount  of  literature.  In  the  third 
step,  results  were  analysed  and  shaped  in  comprehensible  forms. 
It  can  be  concluded  that  not  any  of  the  selected  assessment  tools 
is  judged  as  completely  satisfying  on  all  aspects  of  the  evaluation 
framework.  Although  some  tools  are  already  very  well  developed, 
they  all  have  severe  shortcomings  for  sustainability  assessment  of 
bioenergy,  whether  it  is  from  a  theoretical  point  of  view  and/or  from 
the  practical  side.  Especially  the  absence  of  appropriate  indicators 
and  calculations  of  a  number  of  sustainability  issues,  as  well  as  the 
lack  of  data  and  uncertainty  analysis,  are  often  problematic.  For 
the  moment,  not  any  of  the  selected  tools  could  be  recommended 
to  perform  an  overall  and  integrated  sustainability  assessment  of 
bioenergy  systems.  It  appears  that,  in  the  first  place,  a  good  defini¬ 
tion  of  the  sustainability  problem  is  crucial.  A  different  assessment 
focus  may  lead  to  a  different  choice  of  assessment  tool.  Further¬ 
more,  all  tools  display  particular  advantages  and  disadvantages, 
which  means  that  trade-offs  are  inevitable  and  a  balance  must  be 
found  between  scientific  accuracy  and  pragmatic  decision  making. 
LCA,  for  instance,  performs  undoubtedly  better  on  the  environmen¬ 
tal  part  of  sustainability  while  CBA  is  a  better  choice  with  respect  to 
assessing  economic  issues.  EIA  shows  quite  some  resemblance  to 
LCA  in  terms  of  basic  equations,  and  addresses  even  more  sustain¬ 
ability  issues,  but  the  site-specific  character  of  the  tool  makes  it  very 
time  and  resource  demanding.  C&I  is  probably  the  most  practical 
and  frequently  used  tool,  but  as  is  the  case  for  EIA,  the  assessment 
results  in  a  list  of  parameters  with  varying  units  which  makes  it  dif¬ 
ficult  to  interpret.  EA  seems  to  be  a  worthy  alternative  for  LCA  but 
the  tool  appears  to  be  rather  difficult  to  interpret  and  is  so  far  not 
often  practised  for  sustainability  assessment.  CBA  in  turn  is  praised 
for  its  simple  results  and  its  suitability  for  comparison,  but  mon¬ 
etisation  is  a  source  of  subjectivity  and  uncertainty,  and  reduces 
transparency.  SPA  for  its  part  follows  an  interesting  approach  that 
is  completely  different  from  the  others  looking  at  perturbations  in  a 
system  instead  of  a  products  life  cycle,  but  the  early  developmental 
stage  of  the  tool  impedes  a  generic  use. 

In  addition,  it  can  be  concluded  that  the  various  existing  assess¬ 
ment  tools  are  not  necessarily  complementary,  but  might  be  com¬ 
bined  in  practice  in  the  form  of  a  toolbox.  It  would  thus  be  interest¬ 
ing  to  investigate  the  possibility  of  creating  a  toolbox  that  combines 
the  clear  and  transparent  procedure  of  C&I  with  some  interesting 
procedural  parts  of  EIA  such  as  screening  and  mitigation,  supple¬ 
mented  with  calculation  algorithms  of  LCA  and  CBA  for  respectively 
environmental  and  economic  sustainability  indicators.  Therefore, 
a  more  comprehensive  interdisciplinary  approach  is  necessary  to 
align  the  different  tool  characteristics  and  focuses,  together  with 
additional  research  on  those  sustainability  indicators  for  which  no 
common  calculation  procedure  could  be  developed  yet. 
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Table  Al 

Tool  profile  of  Criteria  &  Indicators  (C&I). 

General 

Tool  description 

Criteria  and  Indicators  (C&I)  are  a  policy  and  management  tool  used  to 
define,  assess  and  monitor  progress  towards  sustainable  management  in 
a  given  area,  over  a  period  of  time  taking  into  consideration  the 
environmental,  social,  economic,  cultural  and  spiritual  needs  of  the  full 
range  of  stakeholder  groups  in  the  areas  concerned  [49]. 

Tool  concept 

After  the  definition  of  goal  and  scope,  a  conceptual  framework  is 
developed  that  is  generic  and  broadly  applicable.  The  framework  is 
divided  into  principles,  criteria,  indicators  and  metrics  and/or  verifiers. 
For  those  issues  for  which  no  indicators  are  currently  available, 
reporting  protocols  can  be  formulated.  Eventually,  indicators  are 
assessed  and  verified. 

Criteria  define  the  essential  elements  against  which  sustainability  is 
assessed.  Each  criterion  relates  to  a  key  element  of  sustainability,  and 
may  be  described  by  one  or  more  indicators.  Indicators  are  parameters 
which  can  be  measured  and  correspond  to  a  particular  criterion.  They 
measure  and  help  monitor  the  current  condition  and  change  over  time 
in  quantitative,  qualitative  and  descriptive  terms  [49].  Metrics  and 
verifiers  provide  additional  data  or  information  that  enhances  the 
specificity  of  the  ease  of  assessment  of  an  indicator. 

Object  of  analysis 

C&I  can  be  used  to  assess  all  kinds  of  systems  on  different  levels, 
although  higher  levels  are  less  frequently  used  (national,  supranational, 
global).  For  example,  in  the  case  of  agricultural  land  use  systems,  the 
system  under  consideration  can  be  the  cropping  system  (plot  level), 
farming  system  (farm  level),  watershed/village  (local  level), 
landscape/district  (regional  level)  or  a  hierarchy  of  different  levels  can 
be  used  [1  ]. 

Type  of  comparison  (between  alternatives,  within  a  studied  system, 
against  a  reference) 

C&I  is  mostly  used  in  an  absolute  way  (no  comparison  of  different 
systems  or  scenarios),  although  C&I  may  also  be  used  to  monitor  a 
system  over  time. 

Standardisation/harmonisation 

There  is  no  standardisation,  but  various  initiatives  have  tried  to  develop 
internationally  harmonised  C&I  frameworks,  for  example  in  sustainable 
forest  management  or  sustainable  palm  oil  production  [50,60]. 
Frequency  of  use 

C&I  is  a  very  frequently  used  tool  because  of  its  ease  of  use,  flexibility 
and  transparency.  The  International  Institute  for  Sustainable 
Development  (USD)  provides  an  overview  of  currently  about  600 
indicator  initiatives  including  the  type  of  initiative,  the  nature  of  public 
involvement,  geographic  scope  and  project  goals  [148,149]. 
Methodological 

Time  frame  (retro-/prospective  and  temporal  boundaries) 

As  a  monitoring  tool,  C&I  is  retrospective,  which  can  be  useful  to  search 
for  preferable  changes  and  to  indicate  sustainability. 

On  indicator  level,  a  difference  can  be  made  between  trend  indicators 
and  state  indicators.  Trend  indicators  capture  the  dynamic  aspect  of 
sustainability  over  time  (e.g.  yield  trends,  depletion  rate)  whereas  state 
indicators  reflect  the  condition  of  the  respective  (eco)system  [1  ]. 

Spatial  scale  (spatial  boundaries) 

C&I  can  be  applied  at  different  levels:  management  level,  regional, 
national  [49]. 

Also  on  indicator  level,  there  are  multiple  spatial  scales  depending  on  the 
indicator  used  [54].  An  example  on  small  spatial  scale  is  the  yield  trend 
and  on  large  spatial  scale  is  the  discount  rate  of  resource  depletion  [1  ]. 
Procedure/algorithm  focus 
Procedure  focus. 

Quantitative/qualitative 
Both  quantitative  and  qualitative. 

Results  (listed  or  aggregated) 

Results  are  listed  with  different  units,  but  an  indicator  might  be  an 
aggregation  of  different  effects  (e.g.  GHG  balance). 
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Table  Al  ( Continued ) 

Weight  scores 

There  is  no  weighing  methodology  included  in  C&I.  In  some  cases,  Multi 
Criteria  Analysis  (MCA)  is  used  to  weight  and  aggregate  results. 
Sensitivity  &  uncertainty 

There  is  no  sensitivity  nor  uncertainty  analysis  included  in  C&I  although 
the  tool  is  subject  to  many  uncertainties  caused  by  the  diversity  of  the 
set  of  criteria  and  indicators,  criteria  content,  indicator  quality,  lack  of 
validity,  poor  data  sets  amongst  others  [144]. 


Table  A2 

Tool  profile  of  Life  Cycle  Assessment  (LCA). 

General 

Tool  description 

Life  Cycle  Assessment  (LCA)  assesses,  in  a  systematic  way,  the  environmental 
issues  and  impacts  of  product  systems,  from  raw  material  acquisition  to  final 
disposal,  in  accordance  with  the  stated  goal  and  scope  [71  ]. 

In  a  bioenergy  perspective,  LCA  is  mostly  used  to  conduct  an  energy  and  GHG 
balance. 

Tool  concept 

There  is  no  single  method  for  conducting  LCA.  Organizations  have  the 
flexibility  to  implement  LCA  as  established  in  the  International  Standard,  in 
accordance  with  the  intended  application  and  the  requirements  of  the 
organization  [72]. 

The  ISO  standard  defines  4  steps:  (1 )  Goal  and  scope  definition  (defined  by  ISO 
14041  [150]);  (2)  Inventory  analysis  (LCI,  defined  by  ISO  14041  [150]);  (3) 
Impact  assessment  including  classification,  characterisation  and  valuation 
(LCIA,  defined  by  ISO  14042  [151]);  (4)  Interpretation  (defined  by  ISO  14043 
[152])  or  improvement  assessment  [117]. 

Object  of  analysis 
Products  and  services. 

Type  of  comparison  (between  alternatives,  within  a  studied  system,  against  a 
reference) 

LCA  is  a  relative  approach,  which  is  structured  around  a  functional  unit.  This 
functional  unit  defines  what  is  being  studied.  Ah  subsequent  analyses  are  then 
relative  to  that  functional  unit,  as  ah  inputs  and  outputs  in  the  LCI  and 
consequently  the  LCIA  profile  are  related  to  the  functional  unit. 
Standardisation/harmonisation 

The  LCA  methodology  is  standardized  by  the  ISO  14040-14043  [71,150-152]. 
Frequency  of  use 

LCA  is  one  of  the  most  frequently  used  systems  analysis  tools. 

Methodological 

Time  frame  (retro-/prospective  and  temporal  boundaries) 

LCA  may  be  used  prospectively  when  used  for  development  of  new  products, 
strategies,  etc.  and  retrospectively  when  used  to  improve  processes  and 
products,  for  purchasing  decisions,  etc.  [95]. 

LCA  takes  the  whole  lifetime  of  a  function  in  account.  The  depth  of  detail  and 
time  frame  of  an  LCA  may  vary  to  a  large  extent,  depending  on  the  goal  and 
scope  definition. 

Time  is  an  important  factor  in  calculations  of  the  annual  life-cycle 
performance  of  biofuels.  Both  energy  inputs  and  outputs  and  emissions  occur 
at  different  points  in  time.  For  example  emissions  from  land  use  due  to 
changes  in  aboveground  biomass  occur  mainly  during  the  short  period  when 
the  land  is  converted  for  cultivation.  The  chosen  time  scale  for  amortization  of 
these  emissions  can  change  the  results  drastically.  The  IPCC  uses  a  default 
value  of  20  years  while  Greenpeace  aims  for  a  10-year  period  [82]. 

Spatial  scale  (spatial  boundaries) 

Since  the  assessment  is  not  site-specific,  there  are  no  geographical  boundaries 
[95]. 

LCA  can  be  used  by  governments,  private  firms,  consumer  organisations,  and 
environmental  groups  as  a  decision  support  tool.  The  scope  of  the  decisions 
covered  by  LCA  ranges  from  broad  management  and  policy  choices  to  specific 
selection  of  product  or  process  characteristics  during  design  [138]. 
Procedure/algorithm  focus 
Algorithm. 

Quantitative/qualitative 

Quantitative. 

Results  (listed  or  aggregated) 

In  the  LCIA  phase,  LCI  results  are  assigned  to  impact  categories.  For  each 
impact  category,  a  life  cycle  impact  category  indicator  is  selected  and  the 
category  indicator  result  (indicator  result)  is  calculated.  It  is  this  list  of 
indicator  results  (LCIA  results  or  LCIA  profile)  that  provides  information  on  the 
environmental  issues  associated  with  the  inputs  and  outputs  of  the  product 
system  [71  ]. 


Table  A2  ( Continued ) 

Weight  scores 

Weighing  of  impacts  is  an  optional  step  in  the  LCIA  phase.  A  number  of 
methods  exist,  divided  in  two  general  groups:  (1)  problem-oriented 
approaches  and  (2)  damage-oriented  methods.  Examples  of  the 
problem-oriented  approaches  are  the  CML-method  and  EDIP  method. 
Examples  of  damage-oriented  methods  are  EPS  2000  and  Eco-Indicator  99. 
The  IMPACT  2002+  method  attempts  to  link  the  problem-  and 
damage-oriented  approaches  in  a  common  framework  [109]. 

Using  the  ISO  standard,  the  valuation  step  is  not  allowed  for  external  studies 
since  there  is  no  scientific  basis  for  reducing  LCA  results  to  a  single  overall 
score  or  number  because  weighing  requires  value  choices. 

Sensitivity  &  uncertainty 

Sensitivity  analysis  is  part  of  the  fourth  LCA  stage  in  ISO  [71  ]. 

Although  various  aspects  such  as  input  parameter  values,  system  boundaries, 
allocation  procedure,  and  fossil  reference  system,  give  rise  to  wide  ranges  of 
results  [83],  no  uncertainty  analysis  is  included  in  the  ISO. 


Table  A3 

Tool  profile  of  Environmental  Impact  Assessment  (EIA). 

General 

Tool  description 

Environmental  Impact  Assessment  (EIA)  is  used  to  evaluate  potential 
environmental  impacts  -  considering  natural,  social  and  economic  issues  - 
that  a  proposed  project  may  have,  with  the  aim  to  reduce  the  negative 
effects  [89]. 

Tool  concept 

More  than  100  countries  worldwide  have  EIA  systems.  They  vary  greatly  in 
terms  of  procedure  and  practice.  The  scope  and  methodologies  of  EIA  have 
evolved  greatly  over  the  past  decade.  Advances  include  consideration  of 
biodiversity  and  climate  change  issues;  increasing  applications  to  policy, 
plans  and  other  strategic  decisions;  and  review  of  trade,  privatisation  and 
structural  adjustment  initiatives  [24]. 

Although  the  amount  of  steps  in  an  EIA  process  may  differ,  in  general  the 
following  steps  can  be  distinguished:  (1)  Screening;  (2)  Scoping;  (3)  Impact 
assessment;  (4)  Mitigation;  (5)  Reporting  (EIS);  (6)  Reviewing;  (7) 
Decision-making;  (8)  Implementation  [24]. 

EIA  is  very  site-specific.  Most  countries  have  lists  of  activities  for  which  EIAs 
are  required  (e.g.  mining  or  major  construction  works).  In  addition,  some 
countries  have  identified  sensitive  environments  (e.g.  estuaries  of  cultural 
heritage  sites)  for  which  EIAs  are  needed  [24]. 

Stakeholder  participation  is  an  important  issue  in  EIA. 

Object  of  analysis 

A  policy,  plan,  programme  or  (mainly)  a  project. 

Type  of  comparison  (between  alternatives,  within  a  studied  system,  against  a 
reference) 

Comparison  is  based  on  the  project  or  site  being  analysed.  There  is  no 
obvious  need  for  a  reference  object,  since  the  purpose  of  the  tool  is  to 
describe  the  effects  a  planned  action  will  have.  However,  EIA  should 
consider  alternatives  to  the  proposed  project  localisation  and  design, 
including  the  zero  alternative  [95]. 

Standardisation/harmonisation 

There  is  an  international  agreement  on  general  procedure  [95].  Often 
countries  have  their  own  procedure. 

Frequency  of  use 

EIA  is  a  well-known  and  frequently  used  tool.  EIAs  are  performed  in  several 
countries  including  some  developing  countries  [95]. 

Methodological 

Time  frame  (retro-/prospective  and  temporal  boundaries) 

Prospective,  foresees  impacts  of  planned  activities  and  improves  long-term 
viability  of  projects  [95]. 

For  a  major  project,  an  EIA  may  take  considerable  time,  manpower  and 
resources.  The  first  2  stages  are  very  important  to  determine  the  required 
extent  and  focus  of  the  EIA  [153]. 

Spatial  scale  (spatial  boundaries) 

(Same  as  time  frame)  The  spatial  boundaries  are  defined  by  the  spatial 
distribution  of  important  direct  and  indirect  impacts  caused  by  the  proposed 
project  or  plan. 

Procedure/algorithm  focus 
Procedure  focus. 

Quantitative/qualitative 
Both  quantitative  and  qualitative. 

Results  (listed  or  aggregated) 

Listed  with  several  different  units.  EIA  reports  tend  to  present  impacts  in 
separate  chapters  for  environmental,  social  and  economic  aspects  [153]. 
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Table  A3  ( Continued ) 


Table  A5  ( Continued ) 


Weight  scores 

No  available  information  on  the  use  of  weight  scores. 

Sensitivity  &  uncertainty 

No  sensitivity  or  uncertainty  analysis  is  included  in  the  EIA  procedure. 


Table  A4 

Tool  profile  of  Cost  Benefit  Analysis  (CBA). 

General 

Tool  description 

Cost  Benefit  Analysis  (CBA)  is  a  method  trying  to  estimate  the  total  impact  of 
a  project  on  society  by  calculating  social  costs  and  benefits.  Environmental 
impacts  are  included  by  valuation  of  these,  converting  them  into  monetary 
terms  [94]. 

Tool  concept 

Steps:  (1)  identification  of  problem  and  alternative  solutions;  (2) 
identification  of  social  costs  and  benefits  for  each  alternative;  (3)  valuation 
of  costs  and  benefits  (usually  in  monetary  terms);  (4)  allocation  of  costs  and 
benefits  over  the  project  time;  (5)  calculation  of  net  present  value  (NPV);  (6) 
ranking  alternatives  by  NPV;  (7)  sensitivity  analysis  should  be  performed; 

(8)  presenting  recommendation  [94]. 

Object  of  analysis 

Projects  (or  sometimes  decisions  on  higher  strategic  levels  e.g.  programmes) 
[95]. 

Type  of  comparison  (between  alternatives,  within  a  studied  system,  against  a 
reference) 

The  zero  alternative  is  used  as  reference  and  different  alternatives  should  be 
compared  [95]. 

Standardisation/harmonisation 

CBA  follows  an  established  methodology.  Rough  guidelines  are  available 
[94]. 

Frequency  of  use 

CBA  is  the  most  well  established  and  utilised  tool  in  economic  decisions 
situations  [95]. 

Methodological 

Time  frame  (retro-/prospective  and  temporal  boundaries) 

Prospective:  estimation  of  future  costs  and  benefits. 

Future  generations  should  be  considered,  which  implies  no  theoretical  time 
boundary  [95].  Preference  of  time  is  considered  through  discounting,  which 
is  a  way  of  including  the  change  of  capital  over  time.  Discounting  is  done  by 
deciding  on  a  discount  rate  defining  present  values  of  future  costs/benefits. 
Spatial  scale  (spatial  boundaries) 

Economic  and  geographical  boundaries  are  set  according  to  the 
development  considered  in  the  CBA  study.  CBA  is  a  partial  economic  analysis 
with  no  feedback  from  other  markets  or  regions  [95]. 

Procedure/algorithm  focus 
Algorithm  focus. 

Quantitative/qualitative 

Quantitative. 

Results  (listed  or  aggregated) 

Results  are  in  monetary  unit.  Costs  and  benefits  that  are  impossible  to 
calculate  in  monetary  terms  may  be  presented  as  intangibles  [95]. 
Sometimes  Multi-criteria  Analysis  (MCA)  or  Social  Impact  Analysis  (SIA)  is 
used  to  count  for  the  intangible  costs  and  benefits  in  the  final  result. 

Weight  scores 

Valuation  of  costs  and  benefits  or  the  use  of  MCA  or  SIA  may  involve  some 
kind  of  weighing. 

Sensitivity  &  uncertainty 

A  sensitivity  analysis  is  included  in  the  procedure  of  the  tool  (step  7)  but  not 
necessarily  performed. 

CBA  is  subject  to  huge  uncertainty  because  of  many  valuations.  Since  the 
result  is  presented  in  an  aggregated  form,  this  uncertainty  may  often  be 
forgotten  [95]. 


Table  A5 

Tool  profile  of  Exergy  Analysis  (EA). 

General 

Tool  description 

Exergy  Analysis  (EA)  is  a  method  that  uses  the  conservation  of  mass 
and  conservation  of  energy  principles  together  with  the  second  law 
of  thermodynamics  for  the  analysis,  design  and  improvement  of 
energy  and  other  systems.  The  exergy  of  an  energy  form  or  a 
substance  is  a  measure  of  its  usefulness  or  quality  or  potential  to 
cause  change.  The  exergy  method  is  a  useful  tool  for  furthering  the 
goal  of  more  efficient  energy-resource  use,  as  it  enables  the 
locations,  types  and  magnitudes  of  wastes  and  losses  to  be  identified 
and  meaningful  efficiencies  to  be  determined  [129]. 


Tool  concept 

Exergy  can  be  accounted  for  both  in  energy  and  material.  It  can 
describe  loss  of  natural  resources  in  physical  terms  [154].  Due  to 
entropy  generation,  the  energy  that  can  be  made  available  from  the 
outputs  (exergy  embodied  in  outputs)  is  less  than  the  energy  that 
can  be  made  available  from  the  inputs  (exergy  embodied  in  inputs), 
although  the  total  energy  of  the  outputs  equals  the  total  energy  of 
the  inputs.  This  quality  degradation  takes  place  in  ah  physical  and 
chemical  processes,  whether  they  take  place  in  the  natural 
ecosystem  (production  of  biomass, . . .)  or  in  the  industrial  ecosystem 
(production,  consumption, . . .)  and  it  is  quantifiable  by  the  loss  of 
exergy.  For  sustainability  assessment  of  products  and  production 
pathways,  three  interacting  sections  are  taken  into  account:  the 
production  and  consumption  chain,  processes  within  the  natural 
ecosystem  and  processes  in  the  industrial  metabolism  [109]. 

Steps:  (1 )  estimation  of  energy  input  to  the  system;  (2)  calculation  of 
exergy  input  (each  energy  contribution  is  multiplied  by  its  energy 
quality  (exergy)  factor);  (3)  estimation  of  energy  output  from  the 
system;  (4)  calculation  of  exergy  output  (as  2);  (5)  output  is  divided 
with  input  and  the  exergy  ratio  is  obtained  [154]. 

Object  of  analysis 

Products,  projects  and  also  economies  [95]. 

Type  of  comparison  (between  alternatives,  within  a  studied  system, 
against  a  reference) 

The  results  of  EAs  are  always  relative  to  the  specified  reference 
environment,  which  in  most  applications  is  modelled  after  the 
actual  local  environment  [106].  An  EA  study  can  serve  to  obtain 
absolute  exergy  values  of  products  and  processes  as  well  as  to 
compare  alternatives  to  find  the  more  exergy  efficient  one  [95,101  ]. 

Standardisation/harmonisation 
No  harmonisation  of  standardisation. 

Frequency  of  use 

To  some  extent  used  in  a  rather  expertised  environment.  Mostly 
used  for  optimisation  of  energy  processes.  In  recent  literature,  the 
exergy  concept  has  been  shown  to  be  a  powerful  tool  in  the 
sustainability  assessment  of  technology  and  in  the  analysis  of 
bioenergy  systems  [102]. 

Methodological 

Time  frame  (retro-/prospective  and  temporal  boundaries) 
Prospectively  for  estimating  the  efficiency  of  potential  developments 
and  retrospectively  for  searching  potential  beneficiary  changes  [95]. 
No  temporal  boundaries.  In  many  cases,  Exergy  Analysis  methods 
take  on  a  life-cycle  perspective,  quantifying  the  cumulated  exergy 
consumption  of  a  product  or  process  from  “cradle  to  grave”  [101  ]. 

Spatial  scale  (spatial  boundaries) 

Geographical  boundaries  are  stated  according  to  the  object  under 
study  [95]. 

Procedure/algorithm  focus 
Algorithm  focus. 

Quantitative/qualitative 

Quantitative. 

Results  (listed  or  aggregated) 

Results  are  aggregated  and  expressed  in  Joule  of  exergy. 

Weight  scores 

No  need  for  weighing  factors  by  using  one  single  unit  that  is 
physically  interpretable  and  stable  with  time  [101  ]. 

Sensitivity  &  uncertainty 

Sensitivity  and  uncertainty  analyses  of  EA  results  were  performed  by 
Dewulf  et  al.  [101,155]. 


Table  A6 

Tool  profile  of  System  Perturbation  Analysis  (SPA). 

General 

Tool  description 

System  Perturbation  Analysis  (SPA)  identifies  the  best  usage  of  limited 
resources  such  as  land,  wood  waste  or  imports,  in  terms  of  fossil  energy 
savings  or  GHG  emissions  within  a  given  system. 

Tool  concept 

A  system  is  defined  in  which  the  possible  scenarios  take  place. 
Conversion  routes  (resource-conversion-product),  utilities,  and  targets 
are  taken  into  account. 

Each  SPA  scenario  is  actually  a  set  of  perturbations  of  resources  and 
targets.  For  each  perturbation  impacts  are  calculated. 

Object  of  analysis 
Products,  projects  and  economies. 
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Table  A6  ( Continued ) 

Type  of  comparison  (between  alternatives,  within  a  studied  system, 
against  a  reference) 

In  an  SPA  study,  alternatives  are  compared  to  a  reference  scenario 
(actual  scenario  or  zero  alternative). 

Standardisation/harmonisation 

No  standardisation  and  very  little  information  on  the  SPA  methodology 
is  available. 

Frequency  of  use 

SPA  is  recently  developed  and  to  date  only  used  by  the  developers 
[115,116]. 

Methodological 

Time  frame  (retro-/prospective  and  temporal  boundaries) 

Prospective,  foresees  impacts  of  possible  projects,  products  or  processes. 
SPA  takes  the  whole  lifetime  of  a  function  into  account. 

Spatial  scale  (spatial  boundaries) 

SPA  considers  a  given  system  where  resources  are  transformed  into 
products  through  a  given  conversion  route.  Examples  exist  already  for 
regional  or  national  systems  [115]  and  for  single  processes  [116]. 
Procedure/algorithm  focus 
Algorithm  focus. 

Quantitative/qualitative 

Quantitative. 

Results  (listed  or  aggregated) 

Listed,  expressed  in  several  different  units. 

Weight  scores 
No  use  of  weight  scores. 

Sensitivity  &  uncertainty 

No  information  was  found  on  any  sensitivity  or  uncertainty  calculations, 
but  in  the  study  of  Delattin  et  al.  [116]  an  error  calculation  was  made. 
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